documentation/winedev-kernel.sgml

   1   <chapter>
   2     <title>Kernel modules</title>
   3     <para>
   4       This section cover the kernel modules. As already stated, Wine
   5       implements the NT architecture, hence provides NTDLL for the
   6       core kernel functions, and KERNEL32, which is the
   7       implementation of the basis of the Win32 subsystem, on top of
   8       NTDLL.
   9     </para>
  10     <sect1 id="ntdll">
  11       <title>NTDLL</title>
  12       <para>
  13         NTDLL provides most of the services you'd expect from a
  14         kernel.
  15       </para>
  16       <para>
  17         Process and thread management are part of them (even if
  18         process management is still mainly done in KERNEL32, unlike
  19         NT). A Windows process runs as a Unix process, and a Windows
  20         thread runs as a Unix thread.
  21       </para>
  22       <para>
  23         Wine also provide fibers (which is the Windows name of
  24         co-routines).
  25       </para>
  26       <para>
  27         Most of the Windows memory handling (Heap, Global and Local
  28         functions, virtual memory...) are easily mapped upon their
  29         Unix equivalents. Note the NTDLL doesn't know about 16 bit
  30         memory, which is only handled in KERNEL32/KRNL386.EXE (and
  31         also the DOS routines).
  32       </para>
  33
  34       <sect2>
  35         <title>File management</title>
  36         <para>
  37           Wine uses some configuration in order to map Windows
  38           filenames (either defined with drive letters, or as UNC
  39           names) to the unix filenames. Wine also uses some
  40           incantation so that most of file related APIs can also
  41           take full unix names. This is handy when passing filenames
  42           on the command line.
  43         </para>
  44         <para>
  45           File handles can be waitable objects, as Windows define
  46           them.
  47         </para>
  48         <para>
  49           Asynchronous I/O is implemented on file handles by
  50           queueing pseudo APC. They are not real APC in the sense
  51           that they have the same priority as the threads in the
  52           considered process (while APCs on NT have normally a
  53           higher priority). These APCs get called when invoking
  54           Wine server (which should lead to correct behavior when the
  55           program ends up waiting on some object - waiting always
  56           implies calling Wine server).
  57         </para>
  58         <para>
  59           FIXME: this should be enhanced and updated to latest work
  60           on FS.
  61         </para>
  62       </sect2>
  63
  64       <sect2>
  65         <title>Synchronization</title>
  66         <para>
  67           Most of the synchronization (between threads or processes)
  68           is done in Wine server, which handles both the waiting
  69           operation (on a single object or a set of objects) and the
  70           signaling of objects.
  71         </para>
  72       </sect2>
  73
  74       <sect2>
  75         <title>Module (DLL) loading</title>
  76         <para>
  77           Wine is able to load any NE and PE module. In all cases,
  78           the module's binary code is directly executed by the
  79           processor.
  80         </para>
  81       </sect2>
  82
  83       <sect2>
  84         <title>Device management</title>
  85         <para>
  86           Wine allows usage a wide variety of devices:
  87           <itemizedlist>
  88             <listitem>
  89               <para>
  90                 Communication ports are mapped to Unix
  91                 communication ports (if they have sufficient
  92                 permissions).
  93               </para>
  94             </listitem>
  95             <listitem>
  96               <para>
  97                 Parallel ports are mapped to Unix parallel ports (if
  98                 they have sufficient permissions).
  99               </para>
 100             </listitem>
 101             <listitem>
 102               <para>
 103                 CDROM: the Windows device I/O control calls are
 104                 mapped onto Unix <function>ioctl()</function>.
 105               </para>
 106             </listitem>
 107             <listitem>
 108               <para>
 109                 Some Win9x VxDs are supported, by rewriting some of
 110                 their internal behavior. But this support is
 111                 limited. Portable programs to Windows NT shouldn't
 112                 need them.
 113               </para>
 114               <para>
 115                 Wine will not support native VxD.
 116               </para>
 117             </listitem>
 118           </itemizedlist>
 119         </para>
 120       </sect2>
 121       <sect2 id="threading">
 122         <title>Multi-threading in Wine</title>
 123
 124         <para>
 125           This section will assume you understand the basics of
 126           multithreading. If not there are plenty of good tutorials
 127           available on the net to get you started.
 128         </para>
 129
 130         <para>
 131           Threading in Wine is somewhat complex due to several
 132           factors. The first is the advanced level of multithreading
 133           support provided by Windows - there are far more threading
 134           related constructs available in Win32 than the Linux
 135           equivalent (pthreads). The second is the need to be able to
 136           map Win32 threads to native Linux threads which provides us
 137           with benefits like having the kernel schedule them without
 138           our intervention. While it's possible to implement threading
 139           entirely without kernel support, doing so is not desirable
 140           on most platforms that Wine runs on.
 141         </para>
 142
 143         <sect3>
 144           <title> Threading support in Win32 </title>
 145
 146           <para>
 147             Win32 is an unusually thread friendly API. Not only is it
 148             entirely thread safe, but it provides many different
 149             facilities for working with threads. These range from the
 150             basics such as starting and stopping threads, to the
 151             extremely complex such as injecting threads into other
 152             processes and COM inter-thread marshalling.
 153           </para>
 154
 155           <para>
 156             One of the primary challenges of writing Wine code
 157             therefore is ensuring that all our DLLs are thread safe,
 158             free of race conditions and so on. This isn't simple -
 159             don't be afraid to ask if you aren't sure whether a piece
 160             of code is thread safe or not!
 161           </para>
 162
 163           <para>
 164             Win32 provides many different ways you can make your code
 165             thread safe however the most common are <emphasis>critical
 166               section</emphasis> and the <emphasis>interlocked
 167               functions</emphasis>. Critical sections are a type of
 168             mutex designed to protect a geographic area of code. If
 169             you don't want multiple threads running in a piece of code
 170             at once, you can protect them with calls to
 171             <function>EnterCriticalSection</function> and
 172             <function>LeaveCriticalSection</function>. The first call
 173             to <function>EnterCriticalSection</function> by a thread
 174             will lock the section and continue without stopping. If
 175             another thread calls it then it will block until the
 176             original thread calls
 177             <function>LeaveCriticalSection</function> again.
 178           </para>
 179
 180           <para>
 181             It is therefore vitally important that if you use critical
 182             sections to make some code thread-safe, that you check
 183             every possible codepath out of the code to ensure that any
 184             held sections are left. Code like this:
 185           </para>
 186
 187           <programlisting>
 188 if (res != ERROR_SUCCESS) return res;
 189           </programlisting>
 190
 191           <para>
 192             is extremely suspect in a function that also contains a
 193             call to <function>EnterCriticalSection</function>. Be
 194             careful.
 195           </para>
 196
 197           <para>
 198             If a thread blocks while waiting for another thread to
 199             leave a critical section, you will see an error from the
 200             <function>RtlpWaitForCriticalSection</function> function,
 201             along with a note of which thread is holding the
 202             lock. This only appears after a certain timeout, normally
 203             a few seconds. It's possible the thread holding the lock
 204             is just being really slow which is why Wine won't
 205             terminate the app like a non-checked build of Windows
 206             would, but the most common cause is that for some reason a
 207             thread forgot to call
 208             <function>LeaveCriticalSection</function>, or died while
 209             holding the lock (perhaps because it was in turn waiting
 210             for another lock). This doesn't just happen in Wine code:
 211             a deadlock while waiting for a critical section could be
 212             due to a bug in the app triggered by a slight difference
 213             in the emulation.
 214           </para>
 215
 216           <para>
 217             Another popular mechanism available is the use of
 218             functions like <function>InterlockedIncrement</function>
 219             and <function>InterlockedExchange</function>. These make
 220             use of native CPU abilities to execute a single
 221             instruction while ensuring any other processors on the
 222             system cannot access memory, and allow you to do common
 223             operations like add/remove/check a variable in thread-safe
 224             code without holding a mutex. These are useful for
 225             reference counting especially in free-threaded (thread
 226             safe) COM objects.
 227           </para>
 228
 229           <para>
 230             Finally, the usage of TLS slots are also popular. TLS
 231             stands for thread-local storage, and is a set of slots
 232             scoped local to a thread which you can store pointers
 233             in. Look on MSDN for the <function>TlsAlloc</function>
 234             function to learn more about the Win32 implementation of
 235             this. Essentially, the contents of a given slot will be
 236             different in each thread, so you can use this to store
 237             data that is only meaningful in the context of a single
 238             thread. On recent versions of Linux the __thread keyword
 239             provides a convenient interface to this functionality - a
 240             more portable API is exposed in the pthread
 241             library. However, these facilities is not used by Wine,
 242             rather, we implement Win32 TLS entirely ourselves.
 243           </para>
 244         </sect3>
 245
 246         <sect3>
 247           <title> SysLevels </title>
 248
 249           <para>
 250             SysLevels are an undocumented Windows-internal
 251             thread-safety system. They are basically critical sections
 252             which must be taken in a particular order. The mechanism
 253             is generic but there are always three syslevels: level 1
 254             is the Win16 mutex, level 2 is the USER mutex and level 3
 255             is the GDI mutex.
 256           </para>
 257
 258           <para>
 259             When entering a syslevel, the code (in
 260             <filename>dlls/kernel/syslevel.c</filename>) will check
 261             that a higher syslevel is not already held and produce an
 262             error if so. This is because it's not legal to enter level
 263             2 while holding level 3 - first, you must leave level 3.
 264           </para>
 265
 266           <para>
 267             Throughout the code you may see calls to
 268             <function>_ConfirmSysLevel()</function> and
 269             <function>_CheckNotSysLevel()</function>. These functions
 270             are essentially assertions about the syslevel states and
 271             can be used to check that the rules have not been
 272             accidentally violated. In particular,
 273             <function>_CheckNotSysLevel()</function> will break
 274             (probably into the debugger) if the check fails. If this
 275             happens the solution is to get a backtrace and find out,
 276             by reading the source of the wine functions called along
 277             the way, how Wine got into the invalid state.
 278           </para>
 279
 280         </sect3>
 281
 282         <sect3>
 283           <title> POSIX threading vs kernel threading </title>
 284
 285           <para>
 286             Wine runs in one of two modes: either pthreads (posix
 287             threading) or kthreads (kernel threading). This section
 288             explains the differences between them. The one that is
 289             used is automatically selected on startup by a small test
 290             program which then execs the correct binary, either
 291             wine-kthread or wine-pthread. On NPTL-enabled systems
 292             pthreads will be used, and on older non-NPTL systems
 293             kthreads is selected.
 294           </para>
 295
 296           <para>
 297             Let's start with a bit of history. Back in the dark ages
 298             when Wines threading support was first implemented a
 299             problem was faced - Windows had much more capable
 300             threading APIs than Linux did. This presented a problem -
 301             Wine works either by reimplementing an API entirely or by
 302             mapping it onto the underlying systems equivalent. How
 303             could Win32 threading be implemented using a library which
 304             did not have all the neeed features? The answer, of
 305             course, was that it couldn't be.
 306           </para>
 307
 308           <para>
 309             On Linux the pthreads interface is used to start, stop and
 310             control threads. The pthreads library in turn is based on
 311             top of so-called "kernel threads" which are created using
 312             the <function>clone(2)</function> syscall. Pthreads
 313             provides a nicer (more portable) interface to this
 314             functionality and also provides APIs for controlling
 315             mutexes. There is a <ulink
 316             url="http://www.llnl.gov/computing/tutorials/pthreads/">
 317             good tutorial on pthreads </ulink> available if you want
 318             to learn more.
 319           </para>
 320
 321           <para>
 322             As pthreads did not provide the necessary semantics to
 323             implement Win32 threading, the decision was made to
 324             implement Win32 threading on top of the underlying kernel
 325             threads by using syscalls like <function>clone</function>
 326             directly. This provided maximum flexibility and allowed a
 327             correct implementation but caused some bad side
 328             effects. Most notably, all the userland Linux APIs assumed
 329             that the user was utilising the pthreads library. Some
 330             only enabled thread safety when they detected that
 331             pthreads was in use - this is true of glibc, for
 332             instance. Worse, pthreads and pure kernel threads had
 333             strange interactions when run in the same process yet some
 334             libraries used by Wine used pthreads internally. Throw in
 335             source code porting using WineLib - where you have both
 336             UNIX and Win32 code in the same process - and chaos was
 337             the result.
 338           </para>
 339
 340           <para>
 341             The solution was simple yet ingenius: Wine would provide
 342             its own implementation of the pthread library
 343             <emphasis>inside</emphasis> its own binary. Due to the
 344             semantics of ELF symbol scoping, this would cause Wines
 345             own implementations to override any implementation loaded
 346             later on (like the real libpthread.so). Therefore, any
 347             calls to the pthread APIs in external libraries would be
 348             linked to Wines instead of the systems pthreads library,
 349             and Wine implemented pthreads by using the standard
 350             Windows threading APIs it in turn implemented itself.
 351           </para>
 352
 353           <para>
 354             As a result, libraries that only became thread-safe in the
 355             presence of a loaded pthreads implementation would now do
 356             so, and any external code that used pthreads would
 357             actually end up creating Win32 threads that Wine was aware
 358             of and controlled. This worked quite nicely for a long
 359             time, even though it required doing some extremely
 360             un-kosher things like overriding internal libc structures
 361             and functions. That is, it worked until NPTL was developed
 362             at which point the underlying thread implementation on
 363             Linux changed dramatically.
 364           </para>
 365
 366           <para>
 367             The fake pthread implementation can be found in
 368             <filename>loader/kthread.c</filename>, which is used to
 369             produce to wine-kthread binary. In contrast,
 370             loader/pthread.c produces the wine-pthread binary which is
 371             used on newer NPTL systems.
 372           </para>
 373
 374           <para>
 375             NPTL is a new threading subsystem for Linux that hugely
 376             improves its performance and flexibility. By allowing
 377             threads to become much more scalable and adding new
 378             pthread APIs, NPTL made Linux competitive with Windows in
 379             the multi-threaded world. Unfortunately it also broke many
 380             assumptions made by Wine (as well as other applications
 381             such as the Sun JVM and RealPlayer) in the process.
 382           </para>
 383
 384           <para>
 385             There was, however, some good news. NPTL made Linux
 386             threading powerful enough that Win32 threads could now be
 387             implemented on top of pthreads like any other normal
 388             application. There would no longer be problems with mixing
 389             win32-kthreads and pthreads created by external libraries,
 390             and no need to override glibc internals. As you can see
 391             from the relative sizes of the
 392             <filename>loader/kthread.c</filename> and
 393             <filename>loader/pthread.c</filename> files, the
 394             difference in code complexity is considerable. NPTL also
 395             made several other semantic changes to things such as
 396             signal delivery so changes were required in many different
 397             places in Wine.
 398           </para>
 399
 400           <para>
 401             On non-Linux systems the threading interface is typically
 402             not powerful enough to replicate the semantics Win32
 403             applications expect and so kthreads with the pthread
 404             overrides are used.
 405           </para>
 406         </sect3>
 407
 408         <sect3>
 409           <title> The Win32 thread environment </title>
 410
 411           <para>
 412             All Win32 code, whether from a native EXE/DLL or in Wine
 413             itself, expects certain constructs to be present in its
 414             environment. This section explores what those constructs
 415             are and how Wine sets them up. The lack of this
 416             environment is one thing that makes it hard to use Wine
 417             code directly from standard Linux applications - in order
 418             to interact with Win32 code a thread must first be
 419             "adopted" by Wine.
 420           </para>
 421
 422           <para>
 423             The first thing Win32 code requires is the
 424             <emphasis>TEB</emphasis> or "Thread Environment
 425             Block". This is an internal (undocumented) Windows
 426             structure associated with every thread which stores a
 427             variety of things such as TLS slots, a pointer to the
 428             threads message queue, the last error code and so on. You
 429             can see the definition of the TEB in
 430             <filename>include/thread.h</filename>, or at least what we
 431             know of it so far. Being internal and subject to change,
 432             the layout of the TEB has had to be reverse engineered
 433             from scratch.
 434           </para>
 435
 436           <para>
 437             A pointer to the TEB is stored in the %fs register and can
 438             be accessed using <function>NtCurrentTeb()</function> from
 439             within Wine code. %fs actually stores a selector, and
 440             setting it therefore requires modifying the processes
 441             local descriptor table (LDT) - the code to do this is in
 442             <filename>lib/wine/ldt.c</filename>.
 443           </para>
 444
 445           <para>
 446             The TEB is required by nearly all Win32 code run in the
 447             Wine environment, as any wineserver RPC will use it, which
 448             in turn implies that any code which could possibly block
 449             (for instance by using a critical section) needs it. The
 450             TEB also holds the SEH exception handler chain as the
 451             first element, so if when disassembling you see code like
 452             this:
 453           </para>
 454
 455           <programlisting> movl %esp, %fs:0 </programlisting>
 456
 457           <para>
 458             ... then you are seeing the program set up an SEH handler
 459             frame. All threads must have at least one SEH entry, which
 460             normally points to the backstop handler which is
 461             ultimately responsible for popping up the all-too-familiar
 462             "This program has performed an illegal operation and will
 463             be terminated" message. On Wine we just drop straight into
 464             the debugger. A full description of SEH is out of the
 465             scope of this section, however there are some good
 466             articles in MSJ if you are interested.
 467           </para>
 468
 469           <para>
 470             All Win32-aware threads must have a wineserver
 471             connection. Many different APIs require the ability to
 472             communicate with the wineserver. In turn, the wineserver
 473             must be aware of Win32 threads in order to be able to
 474             accurately report information to other parts of the program
 475             and do things like route inter-thread messages, dispatch
 476             APCs (asynchronous procedure calls) and so on. Therefore a
 477             part of thread initialization is initializing the thread
 478             serverside. The result is not only correct information in
 479             the server, but a set of file descriptors the thread can use
 480             to communicate with the server - the request fd, reply fd
 481             and wait fd (used for blocking).
 482           </para>
 483         </sect3>
 484       </sect2>
 485     </sect1>
 486
 487     <sect1>
 488       <title>KERNEL Module</title>
 489
 490       <para>
 491         FIXME: Needs some content...
 492       </para>
 493       <sect2 id="consoles">
 494         <title>Consoles in Wine</title>
 495         <para>
 496           As described in the Wine User Guide's CUI section, Wine
 497           manipulates three kinds of "consoles" in order to support
 498           properly the Win32 CUI API.
 499         </para>
 500         <para>
 501           The following table describes the main implementation
 502           differences between the three approaches.
 503           <table>
 504             <title>Function consoles implementation comparison</title>
 505             <tgroup cols="4" align="left">
 506               <thead>
 507                 <row>
 508                   <entry>Function</entry>
 509                   <entry>Bare streams</entry>
 510                   <entry>Wineconsole &amp; user backend</entry>
 511                   <entry>Wineconsole &amp; curses backend</entry>
 512                 </row>
 513               </thead>
 514               <tbody>
 515                 <row>
 516                   <entry>
 517                     Console as a Win32 Object (and associated
 518                     handles)
 519                   </entry>
 520                   <entry>
 521                     No specific Win32 object is used in this
 522                     case. The handles manipulated for the standard
 523                     Win32 streams are in fact "bare handles" to
 524                     their corresponding Unix streams. The mode
 525                     manipulation functions
 526                     (<function>GetConsoleMode</function> /
 527                     <function>SetConsoleMode</function>) are not
 528                     supported.
 529                   </entry>
 530                   <entry>
 531                     Implemented in server, and a specific Winelib
 532                     program (wineconsole) is in charge of the
 533                     rendering and user input. The mode manipulation
 534                     functions behave as expected.
 535                   </entry>
 536                   <entry>
 537                     Implemented in server, and a specific Winelib
 538                     program (wineconsole) is in charge of the
 539                     rendering and user input. The mode manipulation
 540                     functions behave as expected.
 541                   </entry>
 542                 </row>
 543                 <row>
 544                   <entry>
 545                     Inheritance (including handling in
 546                     <function>CreateProcess</function> of
 547                     <constant>CREATE_DETACHED</constant>,
 548                     <constant>CREATE_NEW_CONSOLE</constant> flags).
 549                   </entry>
 550                   <entry>
 551                     Not supported. Every process child of a process
 552                     will inherit the Unix streams, so will also
 553                     inherit the Win32 standard streams.
 554                   </entry>
 555                   <entry>
 556                     Fully supported (each new console creation will
 557                     be handled by the creation of a new USER32
 558                     window)
 559                   </entry>
 560                   <entry>
 561                     Fully supported, except for the creation of a
 562                     new console, which will be rendered on the same
 563                     Unix terminal as the previous one, leading to
 564                     unpredictable results.
 565                   </entry>
 566                 </row>
 567                 <row>
 568                   <entry>
 569                     <function>ReadFile</function> /
 570                     <function>WriteFile</function>
 571                     operations
 572                   </entry>
 573                   <entry>Fully supported</entry>
 574                   <entry>Fully supported</entry>
 575                   <entry>Fully supported</entry>
 576                 </row>
 577                 <row>
 578                   <entry>
 579                     Screen-buffer manipulation (creation, deletion,
 580                     resizing...)
 581                   </entry>
 582                   <entry>Not supported</entry>
 583                   <entry>Fully supported</entry>
 584                   <entry>
 585                     Partly supported (this won't work too well as we
 586                     don't control (so far) the size of underlying
 587                     Unix terminal
 588                   </entry>
 589                 </row>
 590                 <row>
 591                   <entry>
 592                     APIs for reading/writing screen-buffer content,
 593                     cursor position
 594                   </entry>
 595                   <entry>Not supported</entry>
 596                   <entry>Fully supported</entry>
 597                   <entry>Fully supported</entry>
 598                 </row>
 599                 <row>
 600                   <entry>
 601                     APIs for manipulating the rendering window size
 602                   </entry>
 603                   <entry>Not supported</entry>
 604                   <entry>Fully supported</entry>
 605                   <entry>
 606                     Partly supported (this won't work too well as we
 607                     don't control (so far) the size of underlying
 608                     Unix terminal
 609                   </entry>
 610                 </row>
 611                 <row>
 612                   <entry>
 613                     Signaling (in particular, Ctrl-C handling)
 614                   </entry>
 615                   <entry>
 616                     Nothing is done, which means that Ctrl-C will
 617                     generate (as usual) a
 618                     <constant>SIGINT</constant> which will terminate
 619                     the program.
 620                   </entry>
 621                   <entry>
 622                     Partly supported (Ctrl-C behaves as expected,
 623                     however the other Win32 CUI signaling isn't
 624                     properly implemented).
 625                   </entry>
 626                   <entry>
 627                     Partly supported (Ctrl-C behaves as expected,
 628                     however the other Win32 CUI signaling isn't
 629                     properly implemented).
 630                   </entry>
 631                 </row>
 632               </tbody>
 633             </tgroup>
 634           </table>
 635         </para>
 636
 637         <para>
 638           The Win32 objects behind a console can be created in
 639           several occasions:
 640           <itemizedlist>
 641             <listitem>
 642               <para>
 643                 When the program is started from wineconsole, a new
 644                 console object is created and will be used
 645                 (inherited) by the process launched from
 646                 wineconsole.
 647               </para>
 648             </listitem>
 649             <listitem>
 650               <para>
 651                 When a program, which isn't attached to a console,
 652                 calls <function>AllocConsole</function>, Wine then
 653                 launches wineconsole, and attaches the current
 654                 program to this console. In this mode, the USER32
 655                 mode is always selected as Wine cannot tell the
 656                 current state of the Unix console.
 657               </para>
 658             </listitem>
 659           </itemizedlist>
 660         </para>
 661         <para>
 662           Please also note, that starting a child process with the
 663           <constant>CREATE_NEW_CONSOLE</constant> flag, will end-up
 664           calling <function>AllocConsole</function> in the child
 665           process, hence creating a wineconsole with the USER32
 666           backend.
 667         </para>
 668       </sect2>
 669     </sect1>
 670
 671     <sect1 id="initialization">
 672
 673       <title> The Wine initialization process </title>
 674
 675       <para>
 676         Wine has a rather complex startup procedure, so unlike many
 677         programs the best place to begin exploring the code-base is
 678         <emphasis>not</emphasis> in fact at the
 679         <function>main()</function> function but instead at some of the
 680         more straightforward DLLs that exist on the periphery such as
 681         MSI, the widget library (in USER and COMCTL32) etc. The purpose
 682         of this section is to document and explain how Wine starts up
 683         from the moment the user runs "wine myprogram.exe" to the point
 684         at which myprogram gets control.
 685       </para>
 686
 687       <sect2>
 688         <title> First Steps </title>
 689
 690         <para>
 691           The actual wine binary that the user runs does not do very much, in fact it is only
 692           responsible for checking the threading model in use (NPTL vs LinuxThreads) and then invoking
 693           a new binary which performs the next stage in the startup sequence. See the threading chapter
 694           for more information on this check and why it's necessary. You can find this code in
 695           <filename>loader/glibc.c</filename>. The result of this check is an exec of either
 696           wine-pthread or wine-kthread, potentially (on Linux) via
 697           the <emphasis>preloader</emphasis>. We need to use separate binaries here because overriding
 698           the native pthreads library requires us to exploit a property of ELF symbol fixup semantics:
 699           it's not possible to do this without starting a new process.
 700         </para>
 701
 702         <para>
 703           The Wine preloader is found in <filename>loader/preloader.c</filename>, and is required in
 704           order to impose a Win32 style address space layout upon the newly created Win32 process. The
 705           details of what this does is covered in the address space layout chapter. The preloader is a
 706           statically linked ELF binary which is passed the name of the actual Wine binary to run (either
 707           wine-kthread or wine-pthread) along with the arguments the user passed in from the command
 708           line. The preloader is an unusual program: it does not have a main() function. In standard ELF
 709           applications, the entry point is actually at a symbol named _start: this is provided by the
 710           standard gcc infrastructure and normally jumps to <function>__libc_start_main</function> which
 711           initializes glibc before passing control to the main function as defined by the programmer.
 712         </para>
 713
 714         <para>
 715           The preloader takes control direct from the entry point for a few reasons. Firstly, it is
 716           required that glibc is not initialized twice: the result of such behaviour is undefined and
 717           subject to change without notice. Secondly, it's possible that as part of initializing glibc,
 718           the address space layout could be changed - for instance, any call to malloc will initialize a
 719           heap arena which modifies the VM mappings. Finally, glibc does not return to _start at any
 720           point, so by reusing it we avoid the need to recreate the ELF bootstrap stack (env, argv,
 721           auxiliary array etc).
 722         </para>
 723
 724         <para>
 725           The preloader is responsible for two things: protecting important regions of the address
 726           space so the dynamic linker does not map shared libraries into them, and once that is done
 727           loading the real Wine binary off disk, linking it and starting it up. Normally all this is
 728           done automatically by glibc and the kernel but as we intercepted this process by using a
 729           static binary it's up to us to restart the process. The bulk of the code in the preloader is
 730           about loading wine-[pk]thread and ld-linux.so.2 off disk, linking them together, then
 731           starting the dynamic linking process.
 732         </para>
 733
 734         <para>
 735           One of the last things the preloader does before jumping into the dynamic linker is scan the
 736           symbol table of the loaded Wine binary and set the value of a global variable directly: this
 737           is a more efficient way of passing information to the main Wine program than flattening the
 738           data structures into an environment variable or command line parameter then unpacking it on
 739           the other side, but it achieves pretty much the same thing. The global variable set points to
 740           the preload descriptor table, which contains the VMA regions protected by the preloader. This
 741           allows Wine to unmap them once the dynamic linker has been run, so leaving gaps we can
 742           initialize properly later on.
 743         </para>
 744
 745       </sect2>
 746
 747       <sect2>
 748         <title> Starting the emulator </title>
 749
 750         <para>
 751           The process of starting up the emulator itself is mostly one of chaining through various
 752           initializer functions defined in the core libraries and DLLs: libwine, then NTDLL, then kernel32.
 753         </para>
 754
 755         <para>
 756           Both the wine-pthread and wine-kthread binaries share a common <function>main</function>
 757           function, defined in <filename>loader/main.c</filename>, so no matter which binary is selected
 758           after the preloader has run we start here. This passes the information provided by the
 759           preloader into libwine and then calls wine_init, defined
 760           in <filename>libs/wine/loader.c</filename>. This is where the emulation really starts:
 761           <function>wine_init</function> can, with the correct preparation,
 762           be called from programs other than the wine loader itself.
 763         </para>
 764
 765         <para>
 766           <function>wine_init</function> does some very basic setup tasks such as initializing the
 767           debugging infrastructure, yet more address space manipulation (see the information on the
 768           4G/4G VM split in the address space chapter), before loading NTDLL - the core of both Wine and
 769           the Windows NT series - and jumping to the <function>__wine_process_init</function> function defined
 770           in <filename>dlls/ntdll/loader.c</filename>
 771         </para>
 772
 773         <para>
 774           This function is responsible for initializing the primary Win32 environment. In thread_init(),
 775           it sets up the TEB, the wineserver connection for the main thread and the process heap. See
 776           the threading chapter for more information on this.
 777         </para>
 778
 779         <para>
 780           Finally, it loads and jumps to <function>__wine_kernel_init</function> in kernel32.dll: this
 781           is defined in <filename>dlls/kernel32/process.c</filename>. This is where the bulk of the work
 782           is done. The kernel32 initialization code retrieves the startup info for the process from the
 783           server, initializes the registry, sets up the drive mapping system and locale data, then
 784           begins loading the requested application itself. Each process has a STARTUPINFO block that can
 785           be passed into <function>CreateProcess</function> specifying various things like how the first
 786           window should be displayed: this is sent to the new process via the wineserver.
 787         </para>
 788
 789         <para>
 790           After determining the type of file given to Wine by the user (a Win32 EXE file, a Win16 EXE, a
 791           Winelib app etc), the program is loaded into memory (which may involve loading and
 792           initializing other DLLs, the bulk of Wines startup code), before control reaches the end of
 793           <function>__wine_kernel_init</function>. This function ends with the new process stack being
 794           initialized, and start_process being called on the new stack. Nearly there!
 795         </para>
 796
 797         <para>
 798           The final element of initializing Wine is starting the newly loaded program
 799           itself. <function>start_process</function> sets up the SEH backstop handler, calls
 800           <function>LdrInitializeThunk</function> which performs the last part of the process
 801           initialization (such as performing relocations and calling the DllMains with PROCESS_ATTACH),
 802           grabs the entry point of the executable and then on this line:
 803         </para>
 804
 805         <programlisting>
 806 ExitProcess( entry( peb ) );
 807         </programlisting>
 808
 809         <para>
 810           ... jumps to the entry point of the program. At this point the users program is running and
 811           the API provided by Wine is ready to be used. When entry returns,
 812           the <function>ExitProcess</function> API will be used to initialize a graceful shutdown.
 813         </para>
 814       </sect2>
 815     </sect1>
 816
 817     <sect1 id="seh">
 818       <title>Structured Exception Handling</title>
 819
 820       <para>
 821         Structured Exception Handling (or SEH) is an implementation of
 822         exceptions inside the Windows core. It allows code written in
 823         different languages to throw exceptions across DLL boundaries, and
 824         Windows reports various errors like access violations by throwing
 825         them. This section looks at how it works, and how it's implemented
 826         in Wine.
 827       </para>
 828
 829       <sect2>
 830         <title> How SEH works </title>
 831
 832         <para>
 833           SEH is based on embedding EXCEPTION_REGISTRATION_RECORD
 834           structures in the stack. Together they form a linked list rooted
 835           at offset zero in the TEB (see the threading section if you
 836           don't know what this is). A registration record points to a
 837           handler function, and when an exception is thrown the handlers
 838           are executed in turn. Each handler returns a code, and they can
 839           elect to either continue through the handler chain or it can
 840           handle the exception and then restart the program. This is
 841           referred to as unwinding the stack. After each handler is called
 842           it's popped off the chain.
 843         </para>
 844
 845         <para>
 846           Before the system begins unwinding the stack, it runs vectored
 847           handlers. This is an extension to SEH available in Windows XP,
 848           and allows registered functions to get a first chance to watch
 849           or deal with any exceptions thrown in the entire program, from
 850           any thread.
 851         </para>
 852
 853         <para>
 854           A thrown exception is represented by an EXCEPTION_RECORD
 855           structure. It consists of a code, flags, an address and an
 856           arbitrary number of DWORD parameters. Language runtimes can use
 857           these parameters to associate language-specific information with
 858           the exception.
 859         </para>
 860
 861         <para>
 862           Exceptions can be triggered by many things. They can be thrown
 863           explicitly by using the RaiseException API, or they can be
 864           triggered by a crash (ie, translated from a signal). They may be
 865           used internally by a language runtime to implement
 866           language-specific exceptions. They can also be thrown across
 867           DCOM connections.
 868         </para>
 869
 870         <para>
 871           Visual C++ has various extensions to SEH which it uses to
 872           implement, eg, object destruction on stack unwind as well as the
 873           ability to throw arbitrary types. The code for this is in dlls/msvcrt/except.c
 874         </para>
 875
 876       </sect2>
 877
 878       <sect2>
 879         <title> Translating signals to exceptions </title>
 880
 881         <para>
 882           In Windows, compilers are expected to use the system exception
 883           interface, and the kernel itself uses the same interface to
 884           dynamically insert exceptions into a running program. By contrast
 885           on Linux the exception ABI is implemented at the compiler level
 886           (inside GCC and the linker) and the kernel tells a thread of
 887           exceptional events by sending <emphasis>signals</emphasis>. The
 888           language runtime may or may not translate these signals into
 889           native exceptions, but whatever happens the kernel does not care.
 890         </para>
 891
 892         <para>
 893           You may think that if an app crashes, it's game over and it
 894           really shouldn't matter how Wine handles this. It's what you might
 895           intuitively guess, but you'd be wrong. In fact some Windows
 896           programs expect to be able to crash themselves and recover later
 897           without the user noticing, some contain buggy binary-only
 898           components from third parties and use SEH to swallow crashes, and
 899           still others execute priviledged (kernel-level) instructions and
 900           expect it to work. In fact, at least one set of APIs (the
 901           IsBad*Ptr series) can only be implemented by performing an
 902           operation that may crash and returning TRUE if it does, and FALSE
 903           if it doesn't! So, Wine needs to not only implement the SEH
 904           infrastructure but also translate Unix signals into SEH
 905           exceptions.
 906         </para>
 907
 908         <para>
 909           The code to translate signals into exceptions is a part of NTDLL,
 910           and can be found in dlls/ntdll/signal_i386.c. This file sets up
 911           handlers for various signals during Wine startup, and for the ones
 912           that indicate exceptional conditions translates them into
 913           exceptions. Some signals are used by Wine internally and have
 914           nothing to do with SEH.
 915         </para>
 916
 917         <para>
 918           Signal handlers in Wine run on their own stack. Each thread has
 919           its own signal stack which resides 4k after the TEB. This is
 920           important for a couple of reasons. Firstly, because there's no
 921           guarantee that the app thread which triggered the signal has
 922           enough stack space for the Wine signal handling code. In
 923           Windows, if a thread hits the limits of its stack it triggers a
 924           fault on the stack guard page. The language runtime can use this
 925           to grow the stack if it wants to.
 926
 927           <!-- fixme: is it really the language runtime that does this? i
 928                can't find any code in Wine to reallocate the stack on
 929                STATUS_GUARD_PAGE_VIOLATION -->
 930
 931           However, because a guard page violation is just a regular
 932           segfault to the kernel, that would lead to a nested signal
 933           handler and that gets messy really quick so we disallow that in
 934           Wine. Secondly, setting up the exception to throw requires
 935           modifying the stack of the thread which triggered it, which is
 936           quite hard to do when you're still running on it.
 937         </para>
 938
 939         <para>
 940           Windows exceptions typically contain more information than the
 941           Unix standard APIs provide. For instance, a
 942           STATUS_ACCESS_VIOLATION exception (0xC0000005) structure
 943           contains the faulting address, whereas a standard Unix SIGSEGV
 944           just tells the app that it crashed. Usually this information is
 945           passed as an extra parameter to the signal handler, however its
 946           location and contents vary between kernels (BSD, Solaris,
 947           etc). This data is provided in a SIGCONTEXT structure, and on
 948           entry to the signal handler it contains the register state of
 949           the CPU before the signal was sent. Modifying it will cause the
 950           kernel to adjust the context before restarting the thread.
 951         </para>
 952
 953       </sect2>
 954
 955     </sect1>
 956
 957   </chapter>
 958
 959 <!-- Keep this comment at the end of the file
 960 Local variables:
 961 mode: sgml
 962 sgml-parent-document:("wine-devel.sgml" "set" "book" "part" "chapter" "")
 963 End:
 964 -->