documentation/implementation.sgml

   1   <chapter id="implementation">
   2     <title>Low-level Implementation</title>
   3     <para>Details of Wine's Low-level Implementation...</para>
   4
   5 <sect1 id="config-keyboard">
   6 <title>Keyboard</title>
   7
   8 <para>
   9 Wine now needs to know about your keyboard layout. This
  10 requirement comes from a need from many apps to have the
  11 correct scancodes available, since they read these directly,
  12 instead of just taking the characters returned by the X
  13 server. This means that Wine now needs to have a mapping from
  14 X keys to the scancodes these programs expect.
  15 </para>
  16 <para>
  17 On startup, Wine will try to recognize the active X layout by
  18 seeing if it matches any of the defined tables. If it does,
  19 everything is alright. If not, you need to define it.
  20 </para>
  21 <para>
  22 To do this, open the file
  23 <filename>dlls/x11drv/keyboard.c</filename> and take a look
  24 at the existing tables. Make a backup copy of it, especially
  25 if you don't use CVS.
  26 </para>
  27 <para>
  28 What you really would need to do, is find out which scancode
  29 each key needs to generate.  Find it in the
  30 <function>main_key_scan</function> table, which looks like
  31 this:
  32 </para>
  33 <programlisting>
  34 static const int main_key_scan[MAIN_LEN] =
  35 {
  36 /* this is my (102-key) keyboard layout, sorry if it doesn't quite match yours */
  37 0x29,0x02,0x03,0x04,0x05,0x06,0x07,0x08,0x09,0x0A,0x0B,0x0C,0x0D,
  38 0x10,0x11,0x12,0x13,0x14,0x15,0x16,0x17,0x18,0x19,0x1A,0x1B,
  39 0x1E,0x1F,0x20,0x21,0x22,0x23,0x24,0x25,0x26,0x27,0x28,0x2B,
  40 0x2C,0x2D,0x2E,0x2F,0x30,0x31,0x32,0x33,0x34,0x35,
  41 0x56 /* the 102nd key (actually to the right of l-shift) */
  42 };
  43 </programlisting>
  44 <para>
  45 Next, assign each scancode the characters imprinted on the
  46 keycaps. This was done (sort of) for the US 101-key keyboard,
  47 which you can find near the top in
  48 <filename>keyboard.c</filename>. It also shows that if there
  49 is no 102nd key, you can skip that.
  50 </para>
  51 <para>
  52 However, for most international 102-key keyboards, we have
  53 done it easy for you. The scancode layout for these already
  54 pretty much matches the physical layout in the
  55 <function>main_key_scan</function>, so all you need to do is
  56 to go through all the keys that generate characters on your
  57 main keyboard (except spacebar), and stuff those into an
  58 appropriate table. The only exception is that the 102nd key,
  59 which is usually to the left of the first key of the last line
  60 (usually <keycap>Z</keycap>), must be placed on a separate
  61 line after the last line.
  62 </para>
  63 <para>
  64 For example, my Norwegian keyboard looks like this
  65 </para>
  66 <screen>
  67 §  !  "  #  ¤  %  &  /  (  )  =  ?  `  Back-
  68 |  1  2@ 3£ 4$ 5  6  7{ 8[ 9] 0} +  \´ space
  69
  70 Tab Q  W  E  R  T  Y  U  I  O  P  Å  ^
  71                              ¨~
  72                                 Enter
  73 Caps A  S  D  F  G  H  J  K  L  Ø  Æ  *
  74 Lock                                  '
  75
  76 Sh- > Z  X  C  V  B  N  M  ;  :  _  Shift
  77 ift &lt;                      ,  .  -
  78
  79 Ctrl  Alt       Spacebar       AltGr  Ctrl
  80 </screen>
  81 <para>
  82 Note the 102nd key, which is the <keycap>&lt;></keycap> key, to
  83 the left of <keycap>Z</keycap>. The character to the right of
  84 the main character is the character generated by
  85 <keycap>AltGr</keycap>.
  86 </para>
  87 <para>
  88 This keyboard is defined as follows:
  89 </para>
  90 <programlisting>
  91 static const char main_key_NO[MAIN_LEN][4] =
  92 {
  93 "|§","1!","2\"@","3#£","4¤$","5%","6&","7/{","8([","9)]","0=}","+?","\\´",
  94 "qQ","wW","eE","rR","tT","yY","uU","iI","oO","pP","åÅ","¨^~",
  95 "aA","sS","dD","fF","gG","hH","jJ","kK","lL","øØ","æÆ","'*",
  96 "zZ","xX","cC","vV","bB","nN","mM",",;",".:","-_",
  97 "&lt;>"
  98 };
  99 </programlisting>
 100 <para>
 101 Except that " and \ needs to be quoted with a backslash, and
 102 that the 102nd key is on a separate line, it's pretty
 103 straightforward.
 104 </para>
 105 <para>
 106 After you have written such a table, you need to add it to the
 107 <function>main_key_tab[]</function> layout index table. This
 108 will look like this:
 109 </para>
 110 <programlisting>
 111 static struct {
 112 WORD lang, ansi_codepage, oem_codepage;
 113 const char (*key)[MAIN_LEN][4];
 114 } main_key_tab[]={
 115 ...
 116 ...
 117 {MAKELANGID(LANG_NORWEGIAN,SUBLANG_DEFAULT),  1252, 865, &amp;main_key_NO},
 118 ...
 119 </programlisting>
 120 <para>
 121 After you have added your table, recompile Wine and test that
 122 it works. If it fails to detect your table, try running
 123 </para>
 124 <screen>
 125 WINEDEBUG=+key,+keyboard wine > key.log 2>&1
 126       </screen>
 127       <para>
 128         and look in the resulting <filename>key.log</filename> file to
 129         find the error messages it gives for your layout.
 130       </para>
 131       <para>
 132         Note that the <constant>LANG_*</constant> and
 133         <constant>SUBLANG_*</constant> definitions are in
 134         <filename>include/winnls.h</filename>, which you might need to
 135         know to find out which numbers your language is assigned, and
 136         find it in the WINEDEBUG output. The numbers will be
 137         <literal>(SUBLANG * 0x400 + LANG)</literal>, so, for example
 138         the combination <literal>LANG_NORWEGIAN (0x14)</literal> and
 139         <literal>SUBLANG_DEFAULT (0x1)</literal> will be (in hex)
 140         <literal>14 + 1*400 = 414</literal>, so since I'm Norwegian, I
 141         could look for <literal>0414</literal> in the WINEDEBUG output
 142         to find out why my keyboard won't detect.
 143       </para>
 144       <para>
 145         Once it works, submit it to the Wine project. If you use CVS,
 146         you will just have to do
 147       </para>
 148       <screen>
 149 cvs -z3 diff -u dlls/x11drv/keyboard.c > layout.diff
 150       </screen>
 151       <para>
 152         from your main Wine directory, then submit
 153         <filename>layout.diff</filename> to
 154         <email>wine-patches@winehq.org</email> along with a brief note
 155         of what it is.
 156       </para>
 157       <para>
 158         If you don't use CVS, you need to do
 159       </para>
 160       <screen>
 161 diff -u the_backup_file_you_made dlls/x11drv/keyboard.c > layout.diff
 162       </screen>
 163       <para>
 164         and submit it as explained above.
 165       </para>
 166       <para>
 167         If you did it right, it will be included in the next Wine
 168         release, and all the troublesome programs (especially
 169         remote-control programs) and games that use scancodes will
 170         be happily using your keyboard layout, and you won't get those
 171         annoying fixme messages either.
 172       </para>
 173     </sect1>
 174
 175
 176     <sect1 id="undoc-func">
 177       <title>Undocumented APIs</title>
 178
 179         <para>
 180           Some background:  On the i386 class of machines, stack entries are
 181           usually dword (4 bytes) in size, little-endian.  The stack grows
 182           downward in memory.  The stack pointer, maintained in the
 183           <literal>esp</literal> register, points to the last valid entry;
 184           thus, the operation of pushing a value onto the stack involves
 185           decrementing <literal>esp</literal> and then moving the value into
 186           the memory pointed to by <literal>esp</literal>
 187           (i.e., <literal>push p</literal> in assembly resembles
 188           <literal>*(--esp) = p;</literal> in C).  Removing (popping)
 189           values off the stack is the reverse (i.e., <literal>pop p</literal>
 190           corresponds to <literal>p = *(esp++);</literal> in C).
 191         </para>
 192
 193         <para>
 194           In the <literal>stdcall</literal> calling convention, arguments are
 195           pushed onto the stack right-to-left.  For example, the C call
 196           <function>myfunction(40, 20, 70, 30);</function> is expressed in
 197           Intel assembly as:
 198           <screen>
 199     push 30
 200     push 70
 201     push 20
 202     push 40
 203     call myfunction
 204           </screen>
 205           The called function is responsible for removing the arguments
 206           off the stack.  Thus, before the call to myfunction, the
 207           stack would look like:
 208           <screen>
 209              [local variable or temporary]
 210              [local variable or temporary]
 211               30
 212               70
 213               20
 214     esp ->    40
 215           </screen>
 216           After the call returns, it should look like:
 217           <screen>
 218              [local variable or temporary]
 219     esp ->   [local variable or temporary]
 220           </screen>
 221         </para>
 222
 223         <para>
 224           To restore the stack to this state, the called function must know how
 225           many arguments to remove (which is the number of arguments it takes).
 226           This is a problem if the function is undocumented.
 227         </para>
 228
 229         <para>
 230           One way to attempt to document the number of arguments each function
 231           takes is to create a wrapper around that function that detects the
 232           stack offset.  Essentially, each wrapper assumes that the function will
 233           take a large number of arguments.  The wrapper copies each of these
 234           arguments into its stack, calls the actual function, and then calculates
 235           the number of arguments by checking esp before and after the call.
 236         </para>
 237
 238         <para>
 239           The main problem with this scheme is that the function must actually
 240           be called from another program.  Many of these functions are seldom
 241           used.  An attempt was made to aggressively query each function in a
 242           given library (<filename>ntdll.dll</filename>) by passing 64 arguments,
 243           all 0, to each function.  Unfortunately, Windows NT quickly goes to a
 244           blue screen of death, even if the program is run from a
 245           non-administrator account.
 246         </para>
 247
 248         <para>
 249           Another method that has been much more successful is to attempt to
 250           figure out how many arguments each function is removing from the
 251           stack.  This instruction, <literal>ret hhll</literal> (where
 252           <symbol>hhll</symbol> is the number of bytes to remove, i.e. the
 253           number of arguments times 4), contains the bytes
 254           <literal>0xc2 ll hh</literal> in memory.  It is a reasonable
 255           assumption that few, if any, functions take more than 16 arguments;
 256           therefore, simply searching for
 257           <literal>hh == 0 && ll &lt; 0x40</literal>  starting from the
 258           address of a function yields the correct number of arguments most
 259           of the time.
 260         </para>
 261
 262         <para>
 263           Of course, this is not without errors. <literal>ret 00ll</literal>
 264           is not the only instruction that can have the byte sequence
 265           <literal>0xc2 ll 0x0</literal>; for example,
 266           <literal>push 0x000040c2</literal> has the byte sequence
 267           <literal>0x68 0xc2 0x40 0x0 0x0</literal>, which matches
 268           the above.  Properly, the utility should look for this sequence
 269           only on an instruction boundary; unfortunately, finding
 270           instruction boundaries on an i386 requires implementing a full
 271           disassembler -- quite a daunting task.  Besides, the probability
 272           of having such a byte sequence that is not the actual return
 273           instruction is fairly low.
 274         </para>
 275
 276         <para>
 277           Much more troublesome is the non-linear flow of a function.  For
 278           example, consider the following two functions:
 279           <screen>
 280     somefunction1:
 281         jmp  somefunction1_impl
 282
 283     somefunction2:
 284         ret  0004
 285
 286     somefunction1_impl:
 287         ret  0008
 288           </screen>
 289           In this case, we would incorrectly detect both
 290           <function>somefunction1</function> and
 291           <function>somefunction2</function> as taking only a single
 292           argument, whereas <function>somefunction1</function> really
 293           takes two arguments.
 294         </para>
 295
 296         <para>
 297           With these limitations in mind, it is possible to implement more stubs
 298           in Wine and, eventually, the functions themselves.
 299         </para>
 300     </sect1>
 301
 302     <sect1 id="accel-impl">
 303       <title>Accelerators</title>
 304
 305       <para>
 306         There are <emphasis>three</emphasis> differently sized
 307         accelerator structures exposed to the user:
 308       </para>
 309       <orderedlist>
 310         <listitem>
 311           <para>
 312             Accelerators in NE resources.  This is also the internal
 313             layout of the global handle <type>HACCEL</type> (16 and
 314             32) in Windows 95 and Wine. Exposed to the user as Win16
 315             global handles <type>HACCEL16</type> and
 316             <type>HACCEL32</type> by the Win16/Win32 API.
 317             These are 5 bytes long, with no padding:
 318             <programlisting>
 319 BYTE   fVirt;
 320 WORD   key;
 321 WORD   cmd;
 322             </programlisting>
 323           </para>
 324         </listitem>
 325         <listitem>
 326           <para>
 327             Accelerators in PE resources. They are exposed to the user
 328             only by direct accessing PE resources.
 329             These have a size of 8 bytes:
 330           </para>
 331           <programlisting>
 332 BYTE   fVirt;
 333 BYTE   pad0;
 334 WORD   key;
 335 WORD   cmd;
 336 WORD   pad1;
 337           </programlisting>
 338         </listitem>
 339         <listitem>
 340           <para>
 341             Accelerators in the Win32 API.  These are exposed to the
 342             user by the  <function>CopyAcceleratorTable</function>
 343             and  <function>CreateAcceleratorTable</function> functions
 344             in the Win32 API.
 345             These have a size of 6 bytes:
 346           </para>
 347           <programlisting>
 348 BYTE   fVirt;
 349 BYTE   pad0;
 350 WORD   key;
 351 WORD   cmd;
 352           </programlisting>
 353         </listitem>
 354       </orderedlist>
 355
 356       <para>
 357         Why two types of accelerators in the Win32 API? We can only
 358         guess, but my best bet is that the Win32 resource compiler
 359         can/does not handle struct packing. Win32 <type>ACCEL</type>
 360         is defined using <function>#pragma(2)</function> for the
 361         compiler but without any packing for RC, so it will assume
 362         <function>#pragma(4)</function>.
 363       </para>
 364
 365     </sect1>
 366
 367     <sect1 id="hardware-trace">
 368       <title>Doing A Hardware Trace</title>
 369
 370       <para>
 371         The primary reason to do this is to reverse engineer a
 372         hardware device for which you don't have documentation, but
 373         can get to work under Wine.
 374       </para>
 375       <para>
 376         This lot is aimed at parallel port devices, and in particular
 377         parallel port scanners which are now so cheap they are
 378         virtually being given away. The problem is that few
 379         manufactures will release any programming information which
 380         prevents drivers being written for Sane, and the traditional
 381         technique of using DOSemu to produce the traces does not work
 382         as the scanners invariably only have drivers for Windows.
 383       </para>
 384       <para>
 385         Presuming that you have compiled and installed wine the first
 386         thing to do is is to enable direct hardware access to your
 387         parallel port. To do this edit <filename>config</filename>
 388         (usually in <filename>~/.wine/</filename>) and in the
 389         ports section add the following two lines
 390       </para>
 391       <programlisting>
 392 read=0x378,0x379,0x37a,0x37c,0x77a
 393 write=0x378,x379,0x37a,0x37c,0x77a
 394       </programlisting>
 395       <para>
 396         This adds the necessary access required for SPP/PS2/EPP/ECP
 397         parallel port on LPT1. You will need to adjust these number
 398         accordingly if your parallel port is on LPT2 or LPT0.
 399       </para>
 400       <para>
 401         When starting wine use the following command line, where
 402         <literal>XXXX</literal> is the program you need to run in
 403         order to access your scanner, and <literal>YYYY</literal> is
 404         the file your trace will be stored in:
 405       </para>
 406       <programlisting>
 407 WINEDEBUG=+io wine XXXX 2&gt; &gt;(sed 's/^[^:]*:io:[^ ]* //' &gt; YYYY)
 408       </programlisting>
 409       <para>
 410         You will need large amounts of hard disk space (read hundreds
 411         of megabytes if you do a full page scan), and for reasonable
 412         performance a really fast processor and lots of RAM.
 413       </para>
 414       <para>
 415         You will need to postprocess the output into a more manageable
 416         format, using the <command>shrink</command> program. First
 417         you need to compile the source (which is located at the end of
 418         this section):
 419       <programlisting>
 420 cc shrink.c -o shrink
 421       </programlisting>
 422       </para>
 423       <para>
 424         Use the <command>shrink</command> program to reduce the
 425         physical size of the raw log as follows:
 426       </para>
 427       <programlisting>
 428 cat log | shrink &gt; log2
 429       </programlisting>
 430       <para>
 431         The trace has the basic form of
 432       </para>
 433       <programlisting>
 434 XXXX &gt; YY @ ZZZZ:ZZZZ
 435       </programlisting>
 436       <para>
 437         where <literal>XXXX</literal> is the port in hexadecimal being
 438         accessed, <literal>YY</literal> is the data written (or read)
 439         from the port, and <literal>ZZZZ:ZZZZ</literal> is the address
 440         in memory of the instruction that accessed the port. The
 441         direction of the arrow indicates whether the data was written
 442         or read from the port.
 443       </para>
 444       <programlisting>
 445 &gt; data was written to the port
 446 &lt; data was read from the port
 447       </programlisting>
 448       <para>
 449         My basic tip for interpreting these logs is to pay close
 450         attention to the addresses of the IO instructions. Their
 451         grouping and sometimes proximity should reveal the presence of
 452         subroutines in the driver. By studying the different versions
 453         you should be able to work them out. For example consider the
 454         following section of trace from my UMAX Astra 600P
 455       </para>
 456       <programlisting>
 457 0x378 &gt; 55 @ 0297:01ec
 458 0x37a &gt; 05 @ 0297:01f5
 459 0x379 &lt; 8f @ 0297:01fa
 460 0x37a &gt; 04 @ 0297:0211
 461 0x378 &gt; aa @ 0297:01ec
 462 0x37a &gt; 05 @ 0297:01f5
 463 0x379 &lt; 8f @ 0297:01fa
 464 0x37a &gt; 04 @ 0297:0211
 465 0x378 &gt; 00 @ 0297:01ec
 466 0x37a &gt; 05 @ 0297:01f5
 467 0x379 &lt; 8f @ 0297:01fa
 468 0x37a &gt; 04 @ 0297:0211
 469 0x378 &gt; 00 @ 0297:01ec
 470 0x37a &gt; 05 @ 0297:01f5
 471 0x379 &lt; 8f @ 0297:01fa
 472 0x37a &gt; 04 @ 0297:0211
 473 0x378 &gt; 00 @ 0297:01ec
 474 0x37a &gt; 05 @ 0297:01f5
 475 0x379 &lt; 8f @ 0297:01fa
 476 0x37a &gt; 04 @ 0297:0211
 477 0x378 &gt; 00 @ 0297:01ec
 478 0x37a &gt; 05 @ 0297:01f5
 479 0x379 &lt; 8f @ 0297:01fa
 480 0x37a &gt; 04 @ 0297:0211
 481       </programlisting>
 482       <para>
 483         As you can see there is a repeating structure starting at
 484         address <literal>0297:01ec</literal> that consists of four io
 485         accesses on the parallel port. Looking at it the first io
 486         access writes a changing byte to the data port the second
 487         always writes the byte <literal>0x05</literal> to the control
 488         port, then a value which always seems to
 489         <literal>0x8f</literal> is read from the status port at which
 490         point a byte <literal>0x04</literal> is written to the control
 491         port. By studying this and other sections of the trace we can
 492         write a C routine that emulates this, shown below with some
 493         macros to make reading/writing on the parallel port easier to
 494         read.
 495       </para>
 496       <programlisting>
 497 #define r_dtr(x)        inb(x)
 498 #define r_str(x)        inb(x+1)
 499 #define r_ctr(x)        inb(x+2)
 500 #define w_dtr(x,y)      outb(y, x)
 501 #define w_str(x,y)      outb(y, x+1)
 502 #define w_ctr(x,y)      outb(y, x+2)
 503
 504 /* Seems to be sending a command byte to the scanner */
 505 int udpp_put(int udpp_base, unsigned char command)
 506 {
 507     int loop, value;
 508
 509     w_dtr(udpp_base, command);
 510     w_ctr(udpp_base, 0x05);
 511
 512     for (loop=0; loop &lt; 10; loop++)
 513         if ((value = r_str(udpp_base)) & 0x80)
 514         {
 515             w_ctr(udpp_base, 0x04);
 516             return value & 0xf8;
 517         }
 518
 519     return (value & 0xf8) | 0x01;
 520 }
 521       </programlisting>
 522       <para>
 523         For the UMAX Astra 600P only seven such routines exist (well
 524         14 really, seven for SPP and seven for EPP). Whether you
 525         choose to disassemble the driver at this point to verify the
 526         routines is your own choice. If you do, the address from the
 527         trace should help in locating them in the disassembly.
 528       </para>
 529       <para>
 530         You will probably then find it useful to write a script/perl/C
 531         program to analyse the logfile and decode them futher as this
 532         can reveal higher level grouping of the low level routines.
 533         For example from the logs from my UMAX Astra 600P when decoded
 534         further reveal (this is a small snippet)
 535       </para>
 536       <programlisting>
 537 start:
 538 put: 55 8f
 539 put: aa 8f
 540 put: 00 8f
 541 put: 00 8f
 542 put: 00 8f
 543 put: c2 8f
 544 wait: ff
 545 get: af,87
 546 wait: ff
 547 get: af,87
 548 end: cc
 549 start:
 550 put: 55 8f
 551 put: aa 8f
 552 put: 00 8f
 553 put: 03 8f
 554 put: 05 8f
 555 put: 84 8f
 556 wait: ff
 557       </programlisting>
 558       <para>
 559         From this it is easy to see that <varname>put</varname>
 560         routine is often grouped together in five successive calls
 561         sending information to the scanner. Once these are understood
 562         it should be possible to process the logs further to show the
 563         higher level routines in an easy to see format. Once the
 564         highest level format that you can derive from this process is
 565         understood, you then need to produce a series of scans varying
 566         only one parameter between them, so you can discover how to
 567         set the various parameters for the scanner.
 568       </para>
 569
 570       <para>
 571         The following is the <filename>shrink.c</filename> program:
 572       <programlisting>
 573 /* Copyright David Campbell &lt;campbell@torque.net&gt; */
 574 #include &lt;stdio.h&gt;
 575 #include &lt;string.h&gt;
 576
 577 int main (void)
 578 {
 579     char buff[256], lastline[256] = "";
 580     int count = 0;
 581
 582     while (!feof (stdin))
 583     {
 584         fgets (buff, sizeof (buff), stdin);
 585         if (strcmp (buff, lastline))
 586         {
 587             if (count &gt; 1)
 588                 printf ("# Last line repeated %i times #\n", count);
 589             printf ("%s", buff);
 590             strcpy (lastline, buff);
 591             count = 1;
 592         }
 593         else count++;
 594     }
 595     return 0;
 596 }
 597       </programlisting>
 598       </para>
 599     </sect1>
 600
 601   </chapter>
 602
 603 <!-- Keep this comment at the end of the file
 604 Local variables:
 605 mode: sgml
 606 sgml-parent-document:("wine-devel.sgml" "set" "book" "part" "chapter" "")
 607 End:
 608 -->