documentation/address-space.sgml

   1 <chapter id="address-space">
   2   <title> Address space management </title>
   3
   4   <para>
   5     A good understanding of memory layout in Unix and Windows is
   6     required before reading the next section (<xref
   7       linkend="arch-mem"> gives some basic insight).
   8   </para>
   9
  10   <sect1>
  11     <title> Laying out the address space </title>
  12
  13     <para>
  14       Up until about the start of 2004, the Linux address space very much resembled the Windows 9x
  15       layout: the kernel sat in the top gigabyte, the bottom pages were unmapped to catch null
  16       pointer dereferences, and the rest was free. The kernels mmap algorithm was predictable: it
  17       would start by mapping files at low addresses and work up from there.
  18     </para>
  19
  20     <para>
  21       The development of a series of new low level patches violated many of these assumptions, and
  22       resulted in Wine needing to force the Win32 address space layout upon the system. This
  23       section looks at why and how this is done.
  24     </para>
  25
  26     <para>
  27       The exec-shield patch increases security by randomizing the kernels mmap algorithms. Rather
  28       than consistently choosing the same addresses given the same sequence of requests, the kernel
  29       will now choose randomized addresses. Because the Linux dynamic linker (ld-linux.so.2) loads
  30       DSOs into memory by using mmap, this means that DSOs are no longer loaded at predictable
  31       addresses, so making it harder to attack software by using buffer overflows. It also attempts
  32       to relocate certain binaries into a special low area of memory known as the ASCII armor so
  33       making it harder to jump into them when using string based attacks.
  34     </para>
  35
  36     <para>
  37       Prelink is a technology that enhances startup times by precalculating ELF global offset
  38       tables then saving the results inside the native binaries themselves. By grid fitting each
  39       DSO into the address space, the dynamic linker does not have to perform as many relocations
  40       so allowing applications that heavily rely on dynamic linkage to be loaded into memory much
  41       quicker. Complex C++ applications such as Mozilla, OpenOffice and KDE can especially benefit
  42       from this technique.
  43     </para>
  44
  45     <para>
  46       The 4G VM split patch was developed by Ingo Molnar. It gives the Linux kernel its own address
  47       space, thereby allowing processes to access the maximum addressable amount of memory on a
  48       32-bit machine: 4 gigabytes. It allows people with lots of RAM to fully utilise that in any
  49       given process at the cost of performance: the reason behind
  50       giving the kernel a part of each processes address space was to avoid the overhead of switching on
  51       each syscall.
  52     </para>
  53
  54     <para>
  55       Each of these changes alter the address space in a way incompatible with Windows. Prelink and
  56       exec-shield mean that the libraries Wine uses can be placed at any point in the address
  57       space: typically this meant that a library was sitting in the region that the EXE you wanted
  58       to run had to be loaded (remember that unlike DLLs, EXE files cannot be moved around in
  59       memory). The 4G VM split means that programs could receive pointers to the top gigabyte of
  60       address space which some are not prepared for (they may store extra information in the high
  61       bits of a pointer, for instance). In particular, in combination with exec-shield this one is
  62       especially deadly as it's possible the process heap could be allocated beyond
  63       ADDRESS_SPACE_LIMIT which causes Wine initialization to fail.
  64     </para>
  65
  66     <para>
  67       The solution to these problems is for Wine to reserve particular parts of the address space
  68       so that areas that we don't want the system to use will be avoided. We later on
  69       (re/de)allocate those areas as needed. One problem is that some of these mappings are put in
  70       place automatically by the dynamic linker: for instance any libraries that Wine
  71       is linked to (like libc, libwine, libpthread etc) will be mapped into memory before Wine even
  72       gets control. In order to solve that, Wine overrides the default ELF initialization sequence
  73       at a low level and reserves the needed areas by using direct syscalls into the kernel (ie
  74       without linking against any other code to do it) before restarting the standard
  75       initialization and letting the dynamic linker continue. This is referred to as the
  76       preloader and is found in loader/preloader.c.
  77     </para>
  78
  79     <para>
  80       Once the usual ELF boot sequence has been completed, some native libraries may well have been
  81       mapped above the 3gig limit: however, this doesn't matter as 3G is a Windows limit, not a
  82       Linux limit. We still have to prevent the system from allocating anything else above there
  83       (like the heap or other DLLs) though so Wine performs a binary search over the upper gig of
  84       address space in order to iteratively fill in the holes with MAP_NORESERVE mappings so the
  85       address space is allocated but the memory to actually back it is not. This code can be found
  86       in libs/wine/mmap.c:reserve_area.
  87     </para>
  88
  89   </sect1>
  90
  91 </chapter>