1 <chapter id=
"multimedia">
2 <title>Wine and Multimedia
</title>
5 This file contains information about the implementation of the
6 multimedia layer of Wine.
10 The implementation can be found in the dlls/winmm/ directory (and in
11 many of its subdirectories), but also in dlls/msacm/ (for the
12 audio compression/decompression manager) and dlls/msvideo/ (for the
13 video compression/decompression manager).
17 Written by &name-eric-pouech;
<email>&email-eric-pouech;
</email>
18 (Last updated:
02/
16/
2001)
21 <sect1 id=
"mm-overview">
22 <title>Overview
</title>
25 The multimedia stuff is split into
3 layers. The low level (device
26 drivers), mid level (MCI commands) and high level abstraction layers.
27 The low level layer has also some helper DLLs (like the MSACM/MSACM32
28 and MSVIDEO/MSVFW32 pairs).
32 The low level layer may depend on current hardware and OS services
33 (like OSS on Unix). Mid level (MCI) and high level layers must be
34 written independently from the hardware and OS services.
38 There are two specific low level drivers (one for wave input/output,
39 another one for MIDI output only), whose role is:
43 help choosing one low level driver between many
48 add the possibility to convert streams (ie ADPCM =
> PCM)
53 add the possibility to filter a stream (adding echo, equalizer...
54 to a wave stream), or modify the instruments that have to be
62 All of those components are defined as DLLs (one by one).
68 <title>Low level layers
</title>
71 Please note that native low level drivers are not currently supported
72 in Wine, because they either access hardware components or require
73 VxDs to be loaded; Wine does not correctly supports those two so far.
77 The following low level layers are implemented (as built-in DLLs):
81 <title>(Wave form) Audio
</title>
84 MMSYSTEM and WINMM call the real low level audio driver using the
85 wodMessage/widMessage which handles the different requests.
89 <title>OSS implementation
</title>
92 The low level audio driver is currently only implemented for the
93 OpenSoundSystem (OSS) as supplied in the Linux and FreeBSD kernels by
94 <ulink url=
"http://www.4front-tech.com/">4Front Technologies
</ulink>.
95 The presence of this driver is checked by configure (depends on the
96 <sys/soundcard.h
> file). Source code resides in
97 dlls/winmm/wineoss/audio.c.
101 The implementation contains all features commonly used, but has
102 several problems (see TODO list).
106 Note that some Wine specific flag has been added to the wodOpen function,
107 so that the dsound DLL can share the /dev/dsp access. Currently, this
108 only provides mutual exclusion for both DLLs. Future extension could add
109 a virtual mixer between the two output streams.
116 verify all functions for correctness
121 Add virtual mixer between wave-out and dsound interfaces.
130 <title>Other sub systems
</title>
133 No other is available. Could think of Sun Audio, remote audio systems
134 (using X extensions, ...), ALSA, EsounD, ArTs...
145 MMSYSTEM and WINMM call the low level driver functions using the
146 midMessage and the modMessage functions.
150 <title>OSS driver
</title>
153 The low level audio driver is currently only implemented for the
154 OpenSoundSystem (OSS) as supplied in the Linux and FreeBSD kernels by
155 <ulink url=
"http://www.4front-tech.com/">4Front Technologies
</ulink>.
156 The presence of this driver is checked by configure (depends on the
157 <sys/soundcard.h
> file, and also some specific defines because
158 MIDI is not supported on all OSes by OSS). Source code resides in
159 dlls/winmm/wineoss/midi.c
163 Both Midi in and Midi out are provided. The type of MIDI devices
164 supported is external MIDI port (requires an MIDI capable device -
165 keyboard...) and OPL/
2 synthesis (the OPL/
2 patches for all
166 instruments are in midiPatch.c).
173 use better instrument definition for OPL/
2 (midiPatch.c) or use
174 existing instrument definition (from playmidi or kmid) with a
180 have a look at OPL/
3 ?
185 implement asynchronous playback of MidiHdr
190 implement STREAM'ed MidiHdr (question: how shall we share the code
191 between the midiStream functions in MMSYSTEM/WINMM and the code
192 for the low level driver)
197 use a more accurate read mechanism than the one of snooping on
198 timers (like select on fd)
207 <title>Other sub systems
</title>
210 Could support other MIDI implementation for other sub systems (any
215 Could also implement a software synthesizer, either inside Wine or
216 using using MIDI loop back devices in an external program (like
217 timidity). The only trouble is that timidity is GPL'ed...
228 MMSYSTEM and WINMM call the low level driver functions using the
233 <title>OSS implementation
</title>
236 The current implementation uses the OpenSoundSystem mixer, and resides
237 in dlls/winmm/wineoss/mixer.c
244 implement notification mechanism when state of mixer's controls
254 <title>Other sub systems
</title>
260 implement mixing low level drivers for other mixers (ALSA...)
274 The AUX low level driver is the predecessor of the mixer driver
275 (introduced in Win
95).
279 <title>OSS driver
</title>
282 The implementation uses the OSS mixer API, and is incomplete.
289 verify the implementation
294 check with what is done in mixer
299 open question: shall we implement it on top of the low level mixer
312 <title>Wine OSS
</title>
315 All the OSS dependent functions are stored into the WineOSS DLL. It still
316 lack a correct installation scheme (as any multimedia device under Windows),
317 so that all the correct keys are created in the registry. This requires
318 an advanced model since, for example, the number of wave out devices can
319 only be known on the destination system (depends on the sound card driven
320 by the OSS interface). A solution would be to install all the multimedia
321 drivers through the SETUPX DLL; this is not doable yet (the multimedia
322 extension to SETUPX isn't written yet).
328 <title>Joystick
</title>
331 The API consists of the joy* functions found in dlls/winmm/joystick/joystick.c.
332 The implementation currently uses the Linux joystick device driver
333 API. It is lacking support for enhanced joysticks and has not been
341 better support of enhanced joysticks (Linux
2.2 interface is available)
346 support more joystick drivers (like the XInput extension)
351 should load joystick DLL as any other driver (instead of hardcoding)
352 the driver's name, and load it as any low lever driver.
361 <title>Wave mapper (msacm.drv)
</title>
364 The Wave mapper device allows to load on-demand codecs in order to
365 perform software conversion for the types the actual low level driver
366 (hardware). Those codecs are provided through the standard ACM
371 <title>Built-in
</title>
374 A first working implementation for wave out as been provided (wave in
375 exists, but doesn't allow conversion).
379 Wave mapper driver implementation can be found in dlls/winmm/wavemap/
380 directory. This driver heavily relies on MSACM and MSACM32 DLLs which
381 can be found in dlls/msacm and dlls/msacm32. Those DLLs load ACM
382 drivers which provide the conversion to PCM format (which is normally
383 supported by low level drivers). ADPCM, MP3... fit into the category
388 There is currently no built-in ACM driver in Wine, so you must use
389 native ones if you're looking for non PCM playback.
396 check for correctness and robustness
404 <title>Native
</title>
407 Seems to work quite ok (using of course native MSACM/MSACM32 DLLs)
408 Some other testings report some issues while reading back the registry
417 <title>MIDI mapper
</title>
420 Midi mapper allows to map each one of
16 MIDI channels to a specific
421 instrument on an installed sound card. This allows for example to
422 support different MIDI instrument definition (XM, GM...). It also
423 permits to output on a per channel basis to different MIDI renderers.
427 <title>Built-in
</title>
430 A built-in MIDI mapper can be found in dlls/winmm/midimap/. It partly
431 provides the same functionality as the Windows' one. It allows to pick up
432 destination channels (you can map a given channel to a specific playback
433 device channel (see the configuration bits for more details).
440 implement the Midi mapper features (instrument on the fly modification)
441 if it has to be done as under Windows, it required parsing the midi
442 configuration files (didn't find yet the specs)
451 <title>Native
</title>
454 The native midimapper from Win
98 works, but it requires a bunch of
455 keys in the registry which are not part of the Wine source yet.
462 add native midimapper keys to the registry to let it run. This
463 will require proper multimedia driver installation routines.
476 <title>Mid level drivers (MCI)
</title>
479 The mid level drivers are represented by some common API functions,
480 mostly mciSendCommand and mciSendString. See status in chapter
3 for
481 more information. Wine implements several MCI mid level drivers
482 (status is given for both built-in and native implementation):
485 <para>TODO: (apply to all built-in MCI drivers)
489 use MMSYSTEM multitasking caps instead of the home grown
497 <title>CDAUDIO
</title>
500 <title>Built-in
</title>
503 The currently best implementation is the MCI CDAUDIO driver that can
504 be found in dlls/winmm/mcicda/mcicda.c. The implementation is mostly
505 complete, there have been no reports of errors. It makes use of
506 misc/cdrom.c Wine internal cdrom interface.
507 This interface has been ported on Linux, FreeBSD and NetBSD. (Sun
508 should be similar, but are not implemented.)
512 A very small example of a cdplayer consists just of the line
513 mciSendString(
"play cdaudio",NULL,
0,
0);
520 add support for other cdaudio drivers (Solaris...)
525 add support for multiple cdaudio devices (plus a decent
526 configuration scheme)
531 The DLL is not cleanly separated from the NTDLL DLL. The CDROM
532 interface should be exported someway (or stored in a Wine only DLL)
541 <title>Native
</title>
544 Native MCICDA works also correctly... It uses the MSCDEX traps (on int
545 2f). However, some commands (like seeking) seem to be broken.
553 <title>MCIWAVE
</title>
556 <title>Built-in
</title>
559 The implementation is rather complete and can be found in
560 dlls/winmm/mciwave/audio.c. It uses the low level audio API (although
561 not abstracted correctly).
568 The MCI_STATUS command is broken.
578 check for correctness
583 better use of asynchronous playback from low level
588 better implement non waiting command (without the MCI_WAIT flag).
597 <title>Native
</title>
600 Native MCIWAVE works also correctly.
608 <title>MCISEQ (MIDI sequencer)
</title>
611 <title>Built-in
</title>
614 The implementation can be found in dlls/winmm/mciseq/mcimidi.c. Except
615 from the Record command, should be close to completion (except for non
616 blocking commands, as many MCI drivers).
623 implement it correctly
628 finish asynchronous commands (especially for reading/record)
633 better implement non waiting command (without the MCI_WAIT flag).
638 implement the recording features
647 <title>Native
</title>
650 Native MCIMIDI has been working but is currently blocked by scheduling
651 issues (mmTaskXXX no longer work).
657 midiStreamPlay get from time to time an incorrect MidiHdr when
658 using the native MCI sequencer
669 <title>MCIANIM
</title>
672 <title>Built-in
</title>
675 The implementation is in dlls/winmm/mcianim/.
682 implement it, probably using xanim or something similar.
691 <title>Native
</title>
694 Native MCIANIM is reported to work (but requires native video DLLs
695 also, even though the built-in video DLLs start to work correctly).
703 <title>MCIAVI
</title>
706 <title>Built-in
</title>
709 The implementation is in dlls/winmm/mcianim/. Basic features are present,
710 simple playing is available, even if lots remain to be done. It rather
711 heavily relies on MSVIDEO/MSVFW32 DLLs pair to work.
718 finish the implementation
723 fix the audio/video synchronization issue
732 <title>Native
</title>
735 Native MCIAVI is reported to work (but requires native video DLLs
736 also). Some files exhibit some deadlock issues anyway.
746 <title>High level layers
</title>
749 The rest (basically the MMSYSTEM and WINMM DLLs entry points). It also
750 provides the skeleton for the core functionality for multimedia
751 rendering. Note that native MMSYSTEM and WINMM do not currently work
752 under Wine and there is no plan to support them (it would require to
753 also fully support VxD, which is not done yet).
754 Moreover, native DLLs require
16 bit MCI and low level drivers. Wine
755 implements them as
32 bit drivers.
756 MCI and low level drivers can either be
16 or
32 bit for Wine.
763 it seems that some program check what's installed in registry
764 against value returned by drivers. Wine is currently broken
765 regarding this point.
770 add clean-up mechanisms when process detaches from MM DLLs
775 prepare for the
16/
32 big split
780 check thread-safeness for MMSYSTEM and WINMM entry points
785 unicode entry points are badly supported
792 <title>MCI skeleton
</title>
795 Implementation of what is needed to load/unload MCI drivers, and to
796 pass correct information to them. This is implemented in
797 dlls/winmm/mci.c. The mciSendString function uses command strings,
798 which are translated into normal MCI commands as used by
799 mciSendCommand with the help of command tables. The API can be found
800 in dlls/winmm/mmsystem.c and dlls/winmm/mci.c. The functions there
801 (mciOpen,mciSysInfo) handle mid level driver allocation and calls. The
802 implementation is not complete.
806 MCI drivers are seen as regular Wine modules, and can be loaded (with
807 a correct load order between builtin, native, so), as any
808 other DLL. Please note, that MCI drivers module names must bear the
809 .drv extension to be correctly understood.
813 The list of available MCI drivers is obtained as follows:
814 1. key 'mci' in [option] section from .winerc (or wineconf)
815 mci=CDAUDIO:SEQUENCER gives the list of MCI drivers (names, in
816 uppercase only) to be used in Wine.
817 2. This list, when defined, supersedes the mci key in
818 c:\windows\system.ini
822 Note that native VIDEODISC crashes when the module is loaded, which
823 occurs when the MCI procedures are initialized. Make sure that this is
824 not in the list from above. Try adding:
825 mci=CDAUDIO:SEQUENCER:WAVEAUDIO:AVIVIDEO:MPEGVIDEO
826 to the [options] section of the wine config file.
833 correctly handle the MCI_ALL_DEVICE_ID in functions.
838 finish mapping
16 <=
> 32 of MCI structures and commands
843 MCI_SOUND is not handled correctly (should not be sent to MCI
844 driver =
> same behavior as MCI_BREAK)
849 implement auto-open feature (ie, when a string command is issued
850 for a not yet opened device, MCI automatically opens it)
859 <title>MCI multi-tasking
</title>
862 Multi-tasking capabilities used for the MCI drivers are provided in
863 dlls/winmm/mmsystem.c.
870 mmTaskXXX functions are currently broken because the
16 loader does
871 not support binary command lines =
> provide Wine's own mmtask.tsk not
872 using binary command line.
877 the Wine native MCI drivers should use the mmThreadXXX API (but since
878 all built-in MCI drivers are
32 bit, this would require a special
879 flag to mark
32 bit entry points)
888 <title>Timers
</title>
891 It currently uses a service thread, run in the context of the calling
892 process, which should correctly mimic Windows behavior.
899 Check if minimal time is satisfactory for most programs.
904 current implementation may let a timer tick (once) after it has
917 The API consists of the mmio* functions found in dlls/winmm/mmio.c.
918 Seems to work ok in most of the cases. There's some linear/segmented
919 issues with
16 bit code. There are also some bugs when writing MMIO
926 <title>sndPlayXXX functions
</title>
929 Seem to work correctly.
937 <title>Multimedia configuration
</title>
940 Currently, multimedia configuration heavily relies on Win
3.x
945 <title>Drivers
</title>
948 Since all multimedia drivers (MCI, low level ones, ACM drivers,
949 mappers) are, at first, drivers they need to appear in the [mci] or
950 [mci32] section of the system.ini file.
951 Since all drivers are, at first, DLLs, you can choose to load their
952 Wine's (built-in) or Windows (native) version.
961 A default [mci] section (in system.ini) looks like (see the note above
969 waveaudio=mciwave.drv
971 videodisc=mcipionr.drv
977 By default, the list of loadable MCI drivers will be made of those
978 drivers (in the [mci] section).
982 The list of loadable (recognized) MCI drivers can be altered in the
983 [option] section of the wine config file, like:
984 mci=CDAUDIO:SEQUENCER:WAVEAUDIO:AVIVIDEO:MPEGVIDEO
991 use a default registry setting to bypass this (ugly) configuration
1001 <title>Low level drivers
</title>
1004 Configuration of low level drivers is done with the Wine configuration file.
1005 Default keys are provided in winedefault.reg.
1009 The registry keys used here differ from the Windows' one. Using the Windows' one
1010 would require implementing something equivalent to a (real) driver installation.
1011 Even if this would be necessary in a few cases (mainly using MS native multimedia)
1012 modules, there's no real need so far (or it hasn't been run into yet).
1016 See the configuration part of the User's Guide for more details.
1022 <title>Midi mapper
</title>
1025 The Midi mapper configuration is the same as on Windows
9x. Under the key
1027 HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Multimedia\MIDIMap
1029 if the 'UseScheme' value is not set, or is set to a null value, the midi
1030 mapper will always use the driver identified by the 'CurrentInstrument'
1031 value. Note: Wine (for simplicity while installing) allows to define
1032 'CurrentInstrument' as
"#n" (where n is a number), whereas Windows only
1033 allows the real device name here. If UseScheme is set to a non null value,
1034 'CurrentScheme' defines the name of the scheme to map the different channels.
1035 All the schemes are available with keys like
1037 HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\MediaProperties\PrivateProperties\Midi\Schemes\%name_of_scheme%
1039 For every scheme, under this key, will be a sub-key (which name is usually
1040 a two digit index, starting at
00). Its default value is the name of the
1041 output driver, and the value 'Channels' lists all channels (of the
16
1042 standard MIDI ones) which have to be copied to this driver.
1046 To provide enhanced configuration and mapping capabilities, each driver
1047 can define under the key
1049 HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\MediaProperties\PrivateProperties\Midi\Ports\%driver_name%
1051 a link to and .IDF file which allows to remap channels internally (for
1052 example
9 -
> 16), to change instruments identification, event
1053 controllers values. See the source file dlls/winmm/midimap/midimap.c
1054 for the details (this isn't implemented yet).
1062 To be done (use the same mechanism as MCI drivers configuration).
1071 To be done (use the same mechanism as MCI drivers configuration).
1078 <sect1 id=
"mm-arch">
1079 <title>Multimedia architecture
</title>
1082 <title>Windows
9x multimedia architecture
</title>
1086 Kernel space | Client applications
1089 |
16>| |
<32 16>| |
<32 16>| |
<32 16>| |
<32
1091 | +----|-----------|---------|------------|-------+
1092 | | | | | | | WinMM.dll
1093 | | | | | | |
32 bit
1094 | +----|-----------|---------|------------|-------+
1096 | +------+ | |
<16 | | | |
<16 |
1097 | |
16>| | | | | | | |
1099 | | +---------------+---+-------------+-------------+
1100 | | | waveInXXX | | mciXXX | *playSound* |
1101 | | | waveOutXXX | | | mmioXXX |
1102 | | | midiInXXX | | | timeXXX |
1103 | | | midiOutXXX | | | driverXXX |
1104 | | | midiStreamXXX | | | | MMSystem.dll
1105 | | | mixerXXX | | | |
16 bit
1106 +--------+ | | | auxXXX +---+ +---+ mmThread| |
1107 |MMDEVLDR|<------->| joyXXX | Call back | mmTask | |
1108 +--------+ | | +-----------+-----------+---------+-------------+
1110 | | |
16>| |
<16>|
16>| |
<16
1112 +--------+ | | +-------------+ +----------+
1113 | VxD |<------->| *.drv | | mci*.drv |
1114 +--------+ | | +--------------+ +-----------+
1115 | | | msacm.drv | | mciwave |
1116 | | +--------------+ +-----------+
1117 | | | midimap.drv | | mcimidi |
1118 | | +-------------+ +-----------+
1119 | | Low-level drivers | ... | MCI drivers
1123 | +-------------------------------+
1128 The important points to notice are:
1132 all drivers (and most of the core code) is
16 bit
1137 all hardware (or most of it) dependent code reside in the kernel
1138 space (which is not surprising)
1147 <title>Wine multimedia architecture
</title>
1151 Kernel space | Client applications
1154 |
16>| |
<32 16>| |
<32 16>| |
<32 16>| |
<32
1156 | +------+ | | | | | | | |
1157 | |
32/
16>| | | | | | | | |
1158 | | v v v | | v v v v
1159 | | +---------------+---+-------------+-------------+
1160 | | | waveInXXX | | mciXXX | *playSound* |
1161 | | | waveOutXXX | | | mmioXXX | WinMM.dll
1162 | | | midiInXXX | | | timeXXX |
32 bit
1163 | | | midiOutXXX | | | driverXXX |
1164 | | | midiStreamXXX | | | | MMSystem.dll
1165 | | | mixerXXX | | | |
16 bit
1166 | | | auxXXX +---+ +---+ mmThread| |
1167 | | | joyXXX | Call back | mmTask | |
1168 | | +-----------+-----------+---------+-------------+
1170 | |
16>||
<32 |
<16>|
16>||
<32>||
<16
1172 +---------+ | | +-------------+ +----------+
1173 |HW driver|<------->| *.drv | | mci*.drv |
1174 +---------+ | | +--------------+ +-----------+
1175 | | | msacm.drv | | mciwave |
1176 | | +--------------+ +-----------+
1177 | | | midimap.drv | | mcimidi |
1178 | | +-------------+ +-----------+
1179 | | Low-level drivers | ... | MCI drivers
1183 | +-------------------------------+
1188 From the previous drawings, the most noticeable differences are:
1192 low-level drivers can either be
16 or
32 bit
1197 MCI drivers can either be
16 or
32 bit
1202 MMSystem and WinMM will be hosted in a single elfglue library
1207 no link between the MMSystem/WinMM pair on kernel space shall
1208 exist. For example, there will be a low level driver to talk to a
1209 UNIX OSS (Open Sound System) driver
1214 all built-in drivers (low-level and MCI) will be written as
32 bit
1220 all native drivers will be
16 bits drivers
1231 <title>MS ACM Dlls
</title>
1234 <title>Contents
</title>
1240 <title>Status
</title>
1246 <title>Caching
</title>
1249 The MSACM/MSACM32 keeps some data cached for all known ACM
1250 drivers. Under the key
1252 Software\Microsoft\AudioCompressionManager\DriverCache\
<driver
1255 are kept for values:
1259 aFormatTagCache which contains an array of
1260 DWORD. There are two DWORDs per cFormatTags
1261 entry. The first DWORD contains a format tag
1262 value, and the second the associated maximum
1263 size for a WAVEFORMATEX structure.
1264 (Fields dwFormatTag and cbFormatSize from
1270 cFilterTags contains the number of tags supported by the driver
1276 cFormatTags contains the number of tags support
1277 by the driver for conversions.
1282 fdwSupport (the same as the one returned from
1290 The cFilterTags, cFormatTags, fdwSupport are the same
1291 values as the ones returned from acmDriverDetails
1300 <!-- Keep this comment at the end of the file
1303 sgml-parent-document:("wine-devel.sgml" "set" "book" "part" "chapter" "")