[gdb/testsuite] Handle USE_TUI in gdb.tui/corefile-run.exp
[binutils-gdb.git] / gold / icf.cc
blob83610df0a529faa20b0ebc812056853cd782e8a5
1 // icf.cc -- Identical Code Folding.
2 //
3 // Copyright (C) 2009-2023 Free Software Foundation, Inc.
4 // Written by Sriraman Tallam <tmsriram@google.com>.
6 // This file is part of gold.
8 // This program is free software; you can redistribute it and/or modify
9 // it under the terms of the GNU General Public License as published by
10 // the Free Software Foundation; either version 3 of the License, or
11 // (at your option) any later version.
13 // This program is distributed in the hope that it will be useful,
14 // but WITHOUT ANY WARRANTY; without even the implied warranty of
15 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
16 // GNU General Public License for more details.
18 // You should have received a copy of the GNU General Public License
19 // along with this program; if not, write to the Free Software
20 // Foundation, Inc., 51 Franklin Street - Fifth Floor, Boston,
21 // MA 02110-1301, USA.
23 // Identical Code Folding Algorithm
24 // ----------------------------------
25 // Detecting identical functions is done here and the basic algorithm
26 // is as follows. A checksum is computed on each foldable section using
27 // its contents and relocations. If the symbol name corresponding to
28 // a relocation is known it is used to compute the checksum. If the
29 // symbol name is not known the stringified name of the object and the
30 // section number pointed to by the relocation is used. The checksums
31 // are stored as keys in a hash map and a section is identical to some
32 // other section if its checksum is already present in the hash map.
33 // Checksum collisions are handled by using a multimap and explicitly
34 // checking the contents when two sections have the same checksum.
36 // However, two functions A and B with identical text but with
37 // relocations pointing to different foldable sections can be identical if
38 // the corresponding foldable sections to which their relocations point to
39 // turn out to be identical. Hence, this checksumming process must be
40 // done repeatedly until convergence is obtained. Here is an example for
41 // the following case :
43 // int funcA () int funcB ()
44 // { {
45 // return foo(); return goo();
46 // } }
48 // The functions funcA and funcB are identical if functions foo() and
49 // goo() are identical.
51 // Hence, as described above, we repeatedly do the checksumming,
52 // assigning identical functions to the same group, until convergence is
53 // obtained. Now, we have two different ways to do this depending on how
54 // we initialize.
56 // Algorithm I :
57 // -----------
58 // We can start with marking all functions as different and repeatedly do
59 // the checksumming. This has the advantage that we do not need to wait
60 // for convergence. We can stop at any point and correctness will be
61 // guaranteed although not all cases would have been found. However, this
62 // has a problem that some cases can never be found even if it is run until
63 // convergence. Here is an example with mutually recursive functions :
65 // int funcA (int a) int funcB (int a)
66 // { {
67 // if (a == 1) if (a == 1)
68 // return 1; return 1;
69 // return 1 + funcB(a - 1); return 1 + funcA(a - 1);
70 // } }
72 // In this example funcA and funcB are identical and one of them could be
73 // folded into the other. However, if we start with assuming that funcA
74 // and funcB are not identical, the algorithm, even after it is run to
75 // convergence, cannot detect that they are identical. It should be noted
76 // that even if the functions were self-recursive, Algorithm I cannot catch
77 // that they are identical, at least as is.
79 // Algorithm II :
80 // ------------
81 // Here we start with marking all functions as identical and then repeat
82 // the checksumming until convergence. This can detect the above case
83 // mentioned above. It can detect all cases that Algorithm I can and more.
84 // However, the caveat is that it has to be run to convergence. It cannot
85 // be stopped arbitrarily like Algorithm I as correctness cannot be
86 // guaranteed. Algorithm II is not implemented.
88 // Algorithm I is used because experiments show that about three
89 // iterations are more than enough to achieve convergence. Algorithm I can
90 // handle recursive calls if it is changed to use a special common symbol
91 // for recursive relocs. This seems to be the most common case that
92 // Algorithm I could not catch as is. Mutually recursive calls are not
93 // frequent and Algorithm I wins because of its ability to be stopped
94 // arbitrarily.
96 // Caveat with using function pointers :
97 // ------------------------------------
99 // Programs using function pointer comparisons/checks should use function
100 // folding with caution as the result of such comparisons could be different
101 // when folding takes place. This could lead to unexpected run-time
102 // behaviour.
104 // Safe Folding :
105 // ------------
107 // ICF in safe mode folds only ctors and dtors if their function pointers can
108 // never be taken. Also, for X86-64, safe folding uses the relocation
109 // type to determine if a function's pointer is taken or not and only folds
110 // functions whose pointers are definitely not taken.
112 // Caveat with safe folding :
113 // ------------------------
115 // This applies only to x86_64.
117 // Position independent executables are created from PIC objects (compiled
118 // with -fPIC) and/or PIE objects (compiled with -fPIE). For PIE objects, the
119 // relocation types for function pointer taken and a call are the same.
120 // Now, it is not always possible to tell if an object used in the link of
121 // a pie executable is a PIC object or a PIE object. Hence, for pie
122 // executables, using relocation types to disambiguate function pointers is
123 // currently disabled.
125 // Further, it is not correct to use safe folding to build non-pie
126 // executables using PIC/PIE objects. PIC/PIE objects have different
127 // relocation types for function pointers than non-PIC objects, and the
128 // current implementation of safe folding does not handle those relocation
129 // types. Hence, if used, functions whose pointers are taken could still be
130 // folded causing unpredictable run-time behaviour if the pointers were used
131 // in comparisons.
133 // Notes regarding C++ exception handling :
134 // --------------------------------------
136 // It is possible for two sections to have identical text, identical
137 // relocations, but different exception handling metadata (unwind
138 // information in the .eh_frame section, and/or handler information in
139 // a .gcc_except_table section). Thus, if a foldable section is
140 // referenced from a .eh_frame FDE, we must include in its checksum
141 // the contents of that FDE as well as of the CIE that the FDE refers
142 // to. The CIE and FDE in turn probably contain relocations to the
143 // personality routine and LSDA, which are handled like any other
144 // relocation for ICF purposes. This logic is helped by the fact that
145 // gcc with -ffunction-sections puts each function's LSDA in its own
146 // .gcc_except_table.<functionname> section. Given sections for two
147 // functions with nontrivial exception handling logic, we will
148 // determine on the first iteration that their .gcc_except_table
149 // sections are identical and can be folded, and on the second
150 // iteration that their .text and .eh_frame contents (including the
151 // now-merged .gcc_except_table relocations for the LSDA) are
152 // identical and can be folded.
155 // How to run : --icf=[safe|all|none]
156 // Optional parameters : --icf-iterations <num> --print-icf-sections
158 // Performance : Less than 20 % link-time overhead on industry strength
159 // applications. Up to 6 % text size reductions.
161 #include "gold.h"
162 #include "object.h"
163 #include "gc.h"
164 #include "icf.h"
165 #include "symtab.h"
166 #include "libiberty.h"
167 #include "demangle.h"
168 #include "elfcpp.h"
169 #include "int_encoding.h"
171 #include <limits>
173 namespace gold
176 // This function determines if a section or a group of identical
177 // sections has unique contents. Such unique sections or groups can be
178 // declared final and need not be processed any further.
179 // Parameters :
180 // ID_SECTION : Vector mapping a section index to a Section_id pair.
181 // IS_SECN_OR_GROUP_UNIQUE : To check if a section or a group of identical
182 // sections is already known to be unique.
183 // SECTION_CONTENTS : Contains the section's text and relocs to sections
184 // that cannot be folded. SECTION_CONTENTS are NULL
185 // implies that this function is being called for the
186 // first time before the first iteration of icf.
188 static void
189 preprocess_for_unique_sections(const std::vector<Section_id>& id_section,
190 std::vector<bool>* is_secn_or_group_unique,
191 std::vector<std::string>* section_contents)
193 Unordered_map<uint32_t, unsigned int> uniq_map;
194 std::pair<Unordered_map<uint32_t, unsigned int>::iterator, bool>
195 uniq_map_insert;
197 for (unsigned int i = 0; i < id_section.size(); i++)
199 if ((*is_secn_or_group_unique)[i])
200 continue;
202 uint32_t cksum;
203 Section_id secn = id_section[i];
204 section_size_type plen;
205 if (section_contents == NULL)
207 // Lock the object so we can read from it. This is only called
208 // single-threaded from queue_middle_tasks, so it is OK to lock.
209 // Unfortunately we have no way to pass in a Task token.
210 const Task* dummy_task = reinterpret_cast<const Task*>(-1);
211 Task_lock_obj<Object> tl(dummy_task, secn.first);
212 const unsigned char* contents;
213 contents = secn.first->section_contents(secn.second,
214 &plen,
215 false);
216 cksum = xcrc32(contents, plen, 0xffffffff);
218 else
220 const unsigned char* contents_array = reinterpret_cast
221 <const unsigned char*>((*section_contents)[i].c_str());
222 cksum = xcrc32(contents_array, (*section_contents)[i].length(),
223 0xffffffff);
225 uniq_map_insert = uniq_map.insert(std::make_pair(cksum, i));
226 if (uniq_map_insert.second)
228 (*is_secn_or_group_unique)[i] = true;
230 else
232 (*is_secn_or_group_unique)[i] = false;
233 (*is_secn_or_group_unique)[uniq_map_insert.first->second] = false;
238 // For SHF_MERGE sections that use REL relocations, the addend is stored in
239 // the text section at the relocation offset. Read the addend value given
240 // the pointer to the addend in the text section and the addend size.
241 // Update the addend value if a valid addend is found.
242 // Parameters:
243 // RELOC_ADDEND_PTR : Pointer to the addend in the text section.
244 // ADDEND_SIZE : The size of the addend.
245 // RELOC_ADDEND_VALUE : Pointer to the addend that is updated.
247 inline void
248 get_rel_addend(const unsigned char* reloc_addend_ptr,
249 const unsigned int addend_size,
250 uint64_t* reloc_addend_value)
252 switch (addend_size)
254 case 0:
255 break;
256 case 1:
257 *reloc_addend_value =
258 read_from_pointer<8>(reloc_addend_ptr);
259 break;
260 case 2:
261 *reloc_addend_value =
262 read_from_pointer<16>(reloc_addend_ptr);
263 break;
264 case 4:
265 *reloc_addend_value =
266 read_from_pointer<32>(reloc_addend_ptr);
267 break;
268 case 8:
269 *reloc_addend_value =
270 read_from_pointer<64>(reloc_addend_ptr);
271 break;
272 default:
273 gold_unreachable();
277 // This returns the buffer containing the section's contents, both
278 // text and relocs. Relocs are differentiated as those pointing to
279 // sections that could be folded and those that cannot. Only relocs
280 // pointing to sections that could be folded are recomputed on
281 // subsequent invocations of this function.
282 // Parameters :
283 // FIRST_ITERATION : true if it is the first invocation.
284 // FIXED_CACHE : String that stores the portion of the result that
285 // does not change from iteration to iteration;
286 // written if first_iteration is true, read if it's false.
287 // SECN : Section for which contents are desired.
288 // SELF_SECN : Relocations that target this section will be
289 // considered "relocations to self" so that recursive
290 // functions can be folded. Should normally be the
291 // same as `secn` except when processing extra identity
292 // regions.
293 // NUM_TRACKED_RELOCS : Vector reference to store the number of relocs
294 // to ICF sections.
295 // KEPT_SECTION_ID : Vector which maps folded sections to kept sections.
296 // START_OFFSET : Only consider the part of the section at and after
297 // this offset.
298 // END_OFFSET : Only consider the part of the section before this
299 // offset.
301 static std::string
302 get_section_contents(bool first_iteration,
303 std::string* fixed_cache,
304 const Section_id& secn,
305 const Section_id& self_secn,
306 unsigned int* num_tracked_relocs,
307 Symbol_table* symtab,
308 const std::vector<unsigned int>& kept_section_id,
309 section_offset_type start_offset = 0,
310 section_offset_type end_offset =
311 std::numeric_limits<section_offset_type>::max())
313 section_size_type plen;
314 const unsigned char* contents = NULL;
315 if (first_iteration)
316 contents = secn.first->section_contents(secn.second, &plen, false);
318 // The buffer to hold all the contents including relocs. A checksum
319 // is then computed on this buffer.
320 std::string buffer;
321 std::string icf_reloc_buffer;
323 Icf::Reloc_info_list& reloc_info_list =
324 symtab->icf()->reloc_info_list();
326 Icf::Reloc_info_list::iterator it_reloc_info_list =
327 reloc_info_list.find(secn);
329 buffer.clear();
330 icf_reloc_buffer.clear();
332 // Process relocs and put them into the buffer.
334 if (it_reloc_info_list != reloc_info_list.end())
336 Icf::Sections_reachable_info &v =
337 (it_reloc_info_list->second).section_info;
338 // Stores the information of the symbol pointed to by the reloc.
339 const Icf::Symbol_info &s = (it_reloc_info_list->second).symbol_info;
340 // Stores the addend and the symbol value.
341 Icf::Addend_info &a = (it_reloc_info_list->second).addend_info;
342 // Stores the offset of the reloc.
343 const Icf::Offset_info &o = (it_reloc_info_list->second).offset_info;
344 const Icf::Reloc_addend_size_info &reloc_addend_size_info =
345 (it_reloc_info_list->second).reloc_addend_size_info;
346 Icf::Sections_reachable_info::iterator it_v = v.begin();
347 Icf::Symbol_info::const_iterator it_s = s.begin();
348 Icf::Addend_info::iterator it_a = a.begin();
349 Icf::Offset_info::const_iterator it_o = o.begin();
350 Icf::Reloc_addend_size_info::const_iterator it_addend_size =
351 reloc_addend_size_info.begin();
353 for (; it_v != v.end(); ++it_v, ++it_s, ++it_a, ++it_o, ++it_addend_size)
355 Symbol* gsym = *it_s;
356 bool is_section_symbol = false;
358 // Ignore relocations outside the region we were told to look at
359 if (static_cast<section_offset_type>(*it_o) < start_offset
360 || static_cast<section_offset_type>(*it_o) >= end_offset)
361 continue;
363 // A -1 value in the symbol vector indicates a local section symbol.
364 if (gsym == reinterpret_cast<Symbol*>(-1))
366 is_section_symbol = true;
367 gsym = NULL;
370 if (first_iteration
371 && it_v->first != NULL)
373 Symbol_location loc;
374 loc.object = it_v->first;
375 loc.shndx = it_v->second;
376 loc.offset = convert_types<off_t, long long>(it_a->first
377 + it_a->second);
378 // Look through function descriptors
379 parameters->target().function_location(&loc);
380 if (loc.shndx != it_v->second)
382 it_v->second = loc.shndx;
383 // Modify symvalue/addend to the code entry.
384 it_a->first = loc.offset;
385 it_a->second = 0;
389 // ADDEND_STR stores the symbol value and addend and offset,
390 // each at most 16 hex digits long. it_a points to a pair
391 // where first is the symbol value and second is the
392 // addend.
393 char addend_str[50];
395 // It would be nice if we could use format macros in inttypes.h
396 // here but there are not in ISO/IEC C++ 1998.
397 snprintf(addend_str, sizeof(addend_str), "%llx %llx %llx",
398 static_cast<long long>((*it_a).first),
399 static_cast<long long>((*it_a).second),
400 static_cast<unsigned long long>(*it_o - start_offset));
402 // If the symbol pointed to by the reloc is not in an ordinary
403 // section or if the symbol type is not FROM_OBJECT, then the
404 // object is NULL.
405 if (it_v->first == NULL)
407 if (first_iteration)
409 // If the symbol name is available, use it.
410 if (gsym != NULL)
411 buffer.append(gsym->name());
412 // Append the addend.
413 buffer.append(addend_str);
414 buffer.append("@");
416 continue;
419 Section_id reloc_secn(it_v->first, it_v->second);
421 // If this reloc turns back and points to the same section,
422 // like a recursive call, use a special symbol to mark this.
423 if (reloc_secn.first == self_secn.first
424 && reloc_secn.second == self_secn.second)
426 if (first_iteration)
428 buffer.append("R");
429 buffer.append(addend_str);
430 buffer.append("@");
432 continue;
434 Icf::Uniq_secn_id_map& section_id_map =
435 symtab->icf()->section_to_int_map();
436 Icf::Uniq_secn_id_map::iterator section_id_map_it =
437 section_id_map.find(reloc_secn);
438 bool is_sym_preemptible = (gsym != NULL
439 && !gsym->is_from_dynobj()
440 && !gsym->is_undefined()
441 && gsym->is_preemptible());
442 if (!is_sym_preemptible
443 && section_id_map_it != section_id_map.end())
445 // This is a reloc to a section that might be folded.
446 if (num_tracked_relocs)
447 (*num_tracked_relocs)++;
449 char kept_section_str[10];
450 unsigned int secn_id = section_id_map_it->second;
451 snprintf(kept_section_str, sizeof(kept_section_str), "%u",
452 kept_section_id[secn_id]);
453 if (first_iteration)
455 buffer.append("ICF_R");
456 buffer.append(addend_str);
458 icf_reloc_buffer.append(kept_section_str);
459 // Append the addend.
460 icf_reloc_buffer.append(addend_str);
461 icf_reloc_buffer.append("@");
463 else
465 // This is a reloc to a section that cannot be folded.
466 // Process it only in the first iteration.
467 if (!first_iteration)
468 continue;
470 uint64_t secn_flags = (it_v->first)->section_flags(it_v->second);
471 // This reloc points to a merge section. Hash the
472 // contents of this section.
473 if ((secn_flags & elfcpp::SHF_MERGE) != 0
474 && parameters->target().can_icf_inline_merge_sections())
476 uint64_t entsize =
477 (it_v->first)->section_entsize(it_v->second);
478 long long offset = it_a->first;
480 // Handle SHT_RELA and SHT_REL addends. Only one of these
481 // addends exists. When pointing to a merge section, the
482 // addend only matters if it's relative to a section
483 // symbol. In order to unambiguously identify the target
484 // of the relocation, the compiler (and assembler) must use
485 // a local non-section symbol unless Symbol+Addend does in
486 // fact point directly to the target. (In other words,
487 // a bias for a pc-relative reference or a non-zero based
488 // access forces the use of a local symbol, and the addend
489 // is used only to provide that bias.)
490 uint64_t reloc_addend_value = 0;
491 if (is_section_symbol)
493 // Get the SHT_RELA addend. For RELA relocations,
494 // we have the addend from the relocation.
495 reloc_addend_value = it_a->second;
497 // Handle SHT_REL addends.
498 // For REL relocations, we need to fetch the addend
499 // from the section contents.
500 const unsigned char* reloc_addend_ptr =
501 contents + static_cast<unsigned long long>(*it_o);
503 // Update the addend value with the SHT_REL addend if
504 // available.
505 get_rel_addend(reloc_addend_ptr, *it_addend_size,
506 &reloc_addend_value);
508 // Ignore the addend when it is a negative value.
509 // See the comments in Merged_symbol_value::value
510 // in object.h.
511 if (reloc_addend_value < 0xffffff00)
512 offset = offset + reloc_addend_value;
515 section_size_type secn_len;
517 const unsigned char* str_contents =
518 (it_v->first)->section_contents(it_v->second,
519 &secn_len,
520 false) + offset;
521 gold_assert (offset < (long long) secn_len);
523 if ((secn_flags & elfcpp::SHF_STRINGS) != 0)
525 // String merge section.
526 const char* str_char =
527 reinterpret_cast<const char*>(str_contents);
528 switch(entsize)
530 case 1:
532 buffer.append(str_char);
533 break;
535 case 2:
537 const uint16_t* ptr_16 =
538 reinterpret_cast<const uint16_t*>(str_char);
539 unsigned int strlen_16 = 0;
540 // Find the NULL character.
541 while(*(ptr_16 + strlen_16) != 0)
542 strlen_16++;
543 buffer.append(str_char, strlen_16 * 2);
545 break;
546 case 4:
548 const uint32_t* ptr_32 =
549 reinterpret_cast<const uint32_t*>(str_char);
550 unsigned int strlen_32 = 0;
551 // Find the NULL character.
552 while(*(ptr_32 + strlen_32) != 0)
553 strlen_32++;
554 buffer.append(str_char, strlen_32 * 4);
556 break;
557 default:
558 gold_unreachable();
561 else
563 // Use the entsize to determine the length to copy.
564 uint64_t bufsize = entsize;
565 // If entsize is too big, copy all the remaining bytes.
566 if ((offset + entsize) > secn_len)
567 bufsize = secn_len - offset;
568 buffer.append(reinterpret_cast<const
569 char*>(str_contents),
570 bufsize);
572 buffer.append("@");
574 else if (gsym != NULL)
576 // If symbol name is available use that.
577 buffer.append(gsym->name());
578 // Append the addend.
579 buffer.append(addend_str);
580 buffer.append("@");
582 else
584 // Symbol name is not available, like for a local symbol,
585 // use object and section id.
586 buffer.append(it_v->first->name());
587 char secn_id[10];
588 snprintf(secn_id, sizeof(secn_id), "%u",it_v->second);
589 buffer.append(secn_id);
590 // Append the addend.
591 buffer.append(addend_str);
592 buffer.append("@");
598 if (first_iteration)
600 buffer.append("Contents = ");
602 const unsigned char* slice_end =
603 contents + std::min<section_offset_type>(plen, end_offset);
605 if (contents + start_offset < slice_end)
607 buffer.append(reinterpret_cast<const char*>(contents + start_offset),
608 slice_end - (contents + start_offset));
612 // Add any extra identity regions.
613 std::pair<Icf::Extra_identity_list::const_iterator,
614 Icf::Extra_identity_list::const_iterator>
615 extra_range = symtab->icf()->extra_identity_list().equal_range(secn);
616 for (Icf::Extra_identity_list::const_iterator it_ext = extra_range.first;
617 it_ext != extra_range.second; ++it_ext)
619 std::string external_fixed;
620 std::string external_all =
621 get_section_contents(first_iteration, &external_fixed,
622 it_ext->second.section, self_secn,
623 num_tracked_relocs, symtab,
624 kept_section_id, it_ext->second.offset,
625 it_ext->second.offset + it_ext->second.length);
626 buffer.append(external_fixed);
627 icf_reloc_buffer.append(external_all, external_fixed.length(),
628 std::string::npos);
631 if (first_iteration)
633 // Store the section contents that don't change to avoid recomputing
634 // during the next call to this function.
635 *fixed_cache = buffer;
637 else
639 gold_assert(buffer.empty());
641 // Reuse the contents computed in the previous iteration.
642 buffer.append(*fixed_cache);
645 buffer.append(icf_reloc_buffer);
646 return buffer;
649 // This function computes a checksum on each section to detect and form
650 // groups of identical sections. The first iteration does this for all
651 // sections.
652 // Further iterations do this only for the kept sections from each group to
653 // determine if larger groups of identical sections could be formed. The
654 // first section in each group is the kept section for that group.
656 // CRC32 is the checksumming algorithm and can have collisions. That is,
657 // two sections with different contents can have the same checksum. Hence,
658 // a multimap is used to maintain more than one group of checksum
659 // identical sections. A section is added to a group only after its
660 // contents are explicitly compared with the kept section of the group.
662 // Parameters :
663 // ITERATION_NUM : Invocation instance of this function.
664 // NUM_TRACKED_RELOCS : Vector reference to store the number of relocs
665 // to ICF sections.
666 // KEPT_SECTION_ID : Vector which maps folded sections to kept sections.
667 // ID_SECTION : Vector mapping a section to an unique integer.
668 // IS_SECN_OR_GROUP_UNIQUE : To check if a section or a group of identical
669 // sections is already known to be unique.
670 // SECTION_CONTENTS : Store the section's text and relocs to non-ICF
671 // sections.
673 static bool
674 match_sections(unsigned int iteration_num,
675 Symbol_table* symtab,
676 std::vector<unsigned int>* num_tracked_relocs,
677 std::vector<unsigned int>* kept_section_id,
678 const std::vector<Section_id>& id_section,
679 const std::vector<uint64_t>& section_addraligns,
680 std::vector<bool>* is_secn_or_group_unique,
681 std::vector<std::string>* section_contents)
683 Unordered_multimap<uint32_t, unsigned int> section_cksum;
684 std::pair<Unordered_multimap<uint32_t, unsigned int>::iterator,
685 Unordered_multimap<uint32_t, unsigned int>::iterator> key_range;
686 bool converged = true;
688 if (iteration_num == 1)
689 preprocess_for_unique_sections(id_section,
690 is_secn_or_group_unique,
691 NULL);
692 else
693 preprocess_for_unique_sections(id_section,
694 is_secn_or_group_unique,
695 section_contents);
697 std::vector<std::string> full_section_contents;
699 for (unsigned int i = 0; i < id_section.size(); i++)
701 full_section_contents.push_back("");
702 if ((*is_secn_or_group_unique)[i])
703 continue;
705 Section_id secn = id_section[i];
707 // Lock the object so we can read from it. This is only called
708 // single-threaded from queue_middle_tasks, so it is OK to lock.
709 // Unfortunately we have no way to pass in a Task token.
710 const Task* dummy_task = reinterpret_cast<const Task*>(-1);
711 Task_lock_obj<Object> tl(dummy_task, secn.first);
713 std::string this_secn_contents;
714 uint32_t cksum;
715 std::string* this_secn_cache = &((*section_contents)[i]);
716 if (iteration_num == 1)
718 unsigned int num_relocs = 0;
719 this_secn_contents = get_section_contents(true, this_secn_cache,
720 secn, secn, &num_relocs,
721 symtab, (*kept_section_id));
722 (*num_tracked_relocs)[i] = num_relocs;
724 else
726 if ((*kept_section_id)[i] != i)
728 // This section is already folded into something.
729 continue;
731 this_secn_contents = get_section_contents(false, this_secn_cache,
732 secn, secn, NULL,
733 symtab, (*kept_section_id));
736 const unsigned char* this_secn_contents_array =
737 reinterpret_cast<const unsigned char*>(this_secn_contents.c_str());
738 cksum = xcrc32(this_secn_contents_array, this_secn_contents.length(),
739 0xffffffff);
740 size_t count = section_cksum.count(cksum);
742 if (count == 0)
744 // Start a group with this cksum.
745 section_cksum.insert(std::make_pair(cksum, i));
746 full_section_contents[i] = this_secn_contents;
748 else
750 key_range = section_cksum.equal_range(cksum);
751 Unordered_multimap<uint32_t, unsigned int>::iterator it;
752 // Search all the groups with this cksum for a match.
753 for (it = key_range.first; it != key_range.second; ++it)
755 unsigned int kept_section = it->second;
756 if (full_section_contents[kept_section].length()
757 != this_secn_contents.length())
758 continue;
759 if (memcmp(full_section_contents[kept_section].c_str(),
760 this_secn_contents.c_str(),
761 this_secn_contents.length()) != 0)
762 continue;
764 // Check section alignment here.
765 // The section with the larger alignment requirement
766 // should be kept. We assume alignment can only be
767 // zero or positive integral powers of two.
768 uint64_t align_i = section_addraligns[i];
769 uint64_t align_kept = section_addraligns[kept_section];
770 if (align_i <= align_kept)
772 (*kept_section_id)[i] = kept_section;
774 else
776 (*kept_section_id)[kept_section] = i;
777 it->second = i;
778 full_section_contents[kept_section].swap(
779 full_section_contents[i]);
782 converged = false;
783 break;
785 if (it == key_range.second)
787 // Create a new group for this cksum.
788 section_cksum.insert(std::make_pair(cksum, i));
789 full_section_contents[i] = this_secn_contents;
792 // If there are no relocs to foldable sections do not process
793 // this section any further.
794 if (iteration_num == 1 && (*num_tracked_relocs)[i] == 0)
795 (*is_secn_or_group_unique)[i] = true;
798 // If a section was folded into another section that was later folded
799 // again then the former has to be updated.
800 for (unsigned int i = 0; i < id_section.size(); i++)
802 // Find the end of the folding chain
803 unsigned int kept = i;
804 while ((*kept_section_id)[kept] != kept)
806 kept = (*kept_section_id)[kept];
808 // Update every element of the chain
809 unsigned int current = i;
810 while ((*kept_section_id)[current] != kept)
812 unsigned int next = (*kept_section_id)[current];
813 (*kept_section_id)[current] = kept;
814 current = next;
818 return converged;
821 // During safe icf (--icf=safe), only fold functions that are ctors or dtors.
822 // This function returns true if the section name is that of a ctor or a dtor.
824 static bool
825 is_function_ctor_or_dtor(const std::string& section_name)
827 const char* mangled_func_name = strrchr(section_name.c_str(), '.');
828 gold_assert(mangled_func_name != NULL);
829 if ((is_prefix_of("._ZN", mangled_func_name)
830 || is_prefix_of("._ZZ", mangled_func_name))
831 && (is_gnu_v3_mangled_ctor(mangled_func_name + 1)
832 || is_gnu_v3_mangled_dtor(mangled_func_name + 1)))
834 return true;
836 return false;
839 // Iterate through the .eh_frame section that has index
840 // `ehframe_shndx` in `object`, adding entries to extra_identity_list_
841 // that will cause the contents of each FDE and its CIE to be included
842 // in the logical ICF identity of the function that the FDE refers to.
844 bool
845 Icf::add_ehframe_links(Relobj* object, unsigned int ehframe_shndx,
846 Reloc_info& relocs)
848 section_size_type contents_len;
849 const unsigned char* pcontents = object->section_contents(ehframe_shndx,
850 &contents_len,
851 false);
852 const unsigned char* p = pcontents;
853 const unsigned char* pend = pcontents + contents_len;
855 Sections_reachable_info::iterator it_target = relocs.section_info.begin();
856 Sections_reachable_info::iterator it_target_end = relocs.section_info.end();
857 Offset_info::iterator it_offset = relocs.offset_info.begin();
858 Offset_info::iterator it_offset_end = relocs.offset_info.end();
860 // Maps section offset to the length of the CIE defined at that offset.
861 typedef Unordered_map<section_offset_type, section_size_type> Cie_map;
862 Cie_map cies;
864 uint32_t (*read_swap_32)(const unsigned char*);
865 if (object->is_big_endian())
866 read_swap_32 = &elfcpp::Swap<32, true>::readval;
867 else
868 read_swap_32 = &elfcpp::Swap<32, false>::readval;
870 // TODO: The logic for parsing the CIE/FDE framing is copied from
871 // Eh_frame::do_add_ehframe_input_section() and might want to be
872 // factored into a shared helper function.
873 while (p < pend)
875 if (pend - p < 4)
876 return false;
878 unsigned int len = read_swap_32(p);
879 p += 4;
880 if (len == 0)
882 // We should only find a zero-length entry at the end of the
883 // section.
884 if (p < pend)
885 return false;
886 break;
888 // We don't support a 64-bit .eh_frame.
889 if (len == 0xffffffff)
890 return false;
891 if (static_cast<unsigned int>(pend - p) < len)
892 return false;
894 const unsigned char* const pentend = p + len;
896 if (pend - p < 4)
897 return false;
899 unsigned int id = read_swap_32(p);
900 p += 4;
902 if (id == 0)
904 // CIE.
905 cies.insert(std::make_pair(p - pcontents, len - 4));
907 else
909 // FDE.
910 Cie_map::const_iterator it;
911 it = cies.find((p - pcontents) - (id - 4));
912 if (it == cies.end())
913 return false;
915 // Figure out which section this FDE refers into. The word at `p`
916 // is an address, and we expect to see a relocation there. If not,
917 // this FDE isn't ICF-relevant.
918 while (it_offset != it_offset_end
919 && it_target != it_target_end
920 && static_cast<ptrdiff_t>(*it_offset) < (p - pcontents))
922 ++it_offset;
923 ++it_target;
925 if (it_offset != it_offset_end
926 && it_target != it_target_end
927 && static_cast<ptrdiff_t>(*it_offset) == (p - pcontents))
929 // Found a reloc. Add this FDE and its CIE as extra identity
930 // info for the section it refers to.
931 Extra_identity_info rec_fde = {Section_id(object, ehframe_shndx),
932 p - pcontents, len - 4};
933 Extra_identity_info rec_cie = {Section_id(object, ehframe_shndx),
934 it->first, it->second};
935 extra_identity_list_.insert(std::make_pair(*it_target, rec_fde));
936 extra_identity_list_.insert(std::make_pair(*it_target, rec_cie));
940 p = pentend;
943 return true;
946 // This is the main ICF function called in gold.cc. This does the
947 // initialization and calls match_sections repeatedly (thrice by default)
948 // which computes the crc checksums and detects identical functions.
950 void
951 Icf::find_identical_sections(const Input_objects* input_objects,
952 Symbol_table* symtab)
954 unsigned int section_num = 0;
955 std::vector<unsigned int> num_tracked_relocs;
956 std::vector<uint64_t> section_addraligns;
957 std::vector<bool> is_secn_or_group_unique;
958 std::vector<std::string> section_contents;
959 const Target& target = parameters->target();
961 // Decide which sections are possible candidates first.
963 for (Input_objects::Relobj_iterator p = input_objects->relobj_begin();
964 p != input_objects->relobj_end();
965 ++p)
967 // Lock the object so we can read from it. This is only called
968 // single-threaded from queue_middle_tasks, so it is OK to lock.
969 // Unfortunately we have no way to pass in a Task token.
970 const Task* dummy_task = reinterpret_cast<const Task*>(-1);
971 Task_lock_obj<Object> tl(dummy_task, *p);
972 std::vector<unsigned int> eh_frame_ind;
974 for (unsigned int i = 0; i < (*p)->shnum(); ++i)
976 if ((*p)->section_size(i) == 0)
977 continue;
978 const std::string section_name = (*p)->section_name(i);
979 if (!is_section_foldable_candidate(section_name))
981 if (is_prefix_of(".eh_frame", section_name.c_str()))
982 eh_frame_ind.push_back(i);
983 continue;
986 if (!(*p)->is_section_included(i))
987 continue;
988 if (parameters->options().gc_sections()
989 && symtab->gc()->is_section_garbage(*p, i))
990 continue;
991 // With --icf=safe, check if the mangled function name is a ctor
992 // or a dtor. The mangled function name can be obtained from the
993 // section name by stripping the section prefix.
994 if (parameters->options().icf_safe_folding()
995 && !is_function_ctor_or_dtor(section_name)
996 && (!target.can_check_for_function_pointers()
997 || section_has_function_pointers(*p, i)))
999 continue;
1001 this->id_section_.push_back(Section_id(*p, i));
1002 this->section_id_[Section_id(*p, i)] = section_num;
1003 this->kept_section_id_.push_back(section_num);
1004 num_tracked_relocs.push_back(0);
1005 section_addraligns.push_back((*p)->section_addralign(i));
1006 is_secn_or_group_unique.push_back(false);
1007 section_contents.push_back("");
1008 section_num++;
1011 for (std::vector<unsigned int>::iterator it_eh_ind = eh_frame_ind.begin();
1012 it_eh_ind != eh_frame_ind.end(); ++it_eh_ind)
1014 // gc_process_relocs() recorded relocations for this
1015 // section even though we can't fold it. We need to
1016 // use those relocations to associate other foldable
1017 // sections with the FDEs and CIEs that are relevant
1018 // to them, so we can avoid merging sections that
1019 // don't have identical exception-handling behavior.
1021 Section_id sect(*p, *it_eh_ind);
1022 Reloc_info_list::iterator it_rel = this->reloc_info_list().find(sect);
1023 if (it_rel != this->reloc_info_list().end())
1025 if (!add_ehframe_links(*p, *it_eh_ind, it_rel->second))
1027 gold_warning(_("could not parse eh_frame section %s(%s); ICF "
1028 "might not preserve exception handling "
1029 "behavior"),
1030 (*p)->name().c_str(),
1031 (*p)->section_name(*it_eh_ind).c_str());
1037 unsigned int num_iterations = 0;
1039 // Default number of iterations to run ICF is 3.
1040 unsigned int max_iterations = (parameters->options().icf_iterations() > 0)
1041 ? parameters->options().icf_iterations()
1042 : 3;
1044 bool converged = false;
1046 while (!converged && (num_iterations < max_iterations))
1048 num_iterations++;
1049 converged = match_sections(num_iterations, symtab,
1050 &num_tracked_relocs, &this->kept_section_id_,
1051 this->id_section_, section_addraligns,
1052 &is_secn_or_group_unique, &section_contents);
1055 if (parameters->options().print_icf_sections())
1057 if (converged)
1058 gold_info(_("%s: ICF Converged after %u iteration(s)"),
1059 program_name, num_iterations);
1060 else
1061 gold_info(_("%s: ICF stopped after %u iteration(s)"),
1062 program_name, num_iterations);
1065 // Unfold --keep-unique symbols.
1066 for (options::String_set::const_iterator p =
1067 parameters->options().keep_unique_begin();
1068 p != parameters->options().keep_unique_end();
1069 ++p)
1071 const char* name = p->c_str();
1072 Symbol* sym = symtab->lookup(name);
1073 if (sym == NULL)
1075 gold_warning(_("Could not find symbol %s to unfold\n"), name);
1077 else if (sym->source() == Symbol::FROM_OBJECT
1078 && !sym->object()->is_dynamic())
1080 Relobj* obj = static_cast<Relobj*>(sym->object());
1081 bool is_ordinary;
1082 unsigned int shndx = sym->shndx(&is_ordinary);
1083 if (is_ordinary)
1085 this->unfold_section(obj, shndx);
1091 this->icf_ready();
1094 // Unfolds the section denoted by OBJ and SHNDX if folded.
1096 void
1097 Icf::unfold_section(Relobj* obj, unsigned int shndx)
1099 Section_id secn(obj, shndx);
1100 Uniq_secn_id_map::iterator it = this->section_id_.find(secn);
1101 if (it == this->section_id_.end())
1102 return;
1103 unsigned int section_num = it->second;
1104 unsigned int kept_section_id = this->kept_section_id_[section_num];
1105 if (kept_section_id != section_num)
1106 this->kept_section_id_[section_num] = section_num;
1109 // This function determines if the section corresponding to the
1110 // given object and index is folded based on if the kept section
1111 // is different from this section.
1113 bool
1114 Icf::is_section_folded(Relobj* obj, unsigned int shndx)
1116 Section_id secn(obj, shndx);
1117 Uniq_secn_id_map::iterator it = this->section_id_.find(secn);
1118 if (it == this->section_id_.end())
1119 return false;
1120 unsigned int section_num = it->second;
1121 unsigned int kept_section_id = this->kept_section_id_[section_num];
1122 return kept_section_id != section_num;
1125 // This function returns the folded section for the given section.
1127 Section_id
1128 Icf::get_folded_section(Relobj* dup_obj, unsigned int dup_shndx)
1130 Section_id dup_secn(dup_obj, dup_shndx);
1131 Uniq_secn_id_map::iterator it = this->section_id_.find(dup_secn);
1132 gold_assert(it != this->section_id_.end());
1133 unsigned int section_num = it->second;
1134 unsigned int kept_section_id = this->kept_section_id_[section_num];
1135 Section_id folded_section = this->id_section_[kept_section_id];
1136 return folded_section;
1139 } // End of namespace gold.