3 <h1>API Versioning in AVM
</h1>
8 14-Sep-
2009 jodyer@adobe.com updated to reflect new bitmask mechanics
9 22-Jul-
2009 jodyer@adobe.com initial draft
15 <P><span class=
"pcounter"></span>
16 API versioning in AVM allows code compiled against one version of a built-in API
17 to see exactly the same set of bindings when run on a different, compatible
18 version of that API. A compatible version is one that has zero or more public
19 names added to it, and none removed from it. Thus, in AVM, APIs may add global
20 and class members without the risk of breaking existing ABC programs. The
21 versioning mechanism described here supports simultaneously executing ABCs of
26 <P><span class=
"pcounter"></span>
27 Logically, a version can be thought of as a set of bindings. There is a partial
28 order of versions such that every version is a subset of itself and zero or more
29 other versions. A version is said to be compatible with every other version of
30 which it is a superset including itself.
32 <P><span class=
"pcounter"></span>
33 By way of example, imagine there exists a runtime called FR. It supports the
34 evolution of two coexisting profiles over time. One product supports program
35 execution in one of the the various browsers via a plugin (let's call it FP). The
36 other product supports the execution of standalone applications and is compatible
37 with the web product (let's refer to it as AR). At any point in time the latest
38 version of AR is larger than (includes a superset of names of) FP. And
39 over time each new release of each products is larger than all previous releases
42 <P><span class=
"pcounter"></span>
43 Now imagine that there exists versions of these two products as described
44 by the following relations:
57 <P><span class=
"pcounter"></span>From these relations we can see, for example, that a binding introduced in
58 FP_9_0 will be in every other version; a binding introduced in AR_1_0 is not in
59 FP_9_0 (or any other FP version, for that matter) but is in AR_1_5; and a binding
60 introduced in AR_2_0 is only in AR_2_0.
62 <P><span class=
"pcounter"></span>
63 In source code, each built-in (library code) definition may be annotated with a
64 metadata attribute that indicates the version or versions in which it was
65 introduced. If no metadata occurs on a definition, it is assumed to have been
66 introduced in the smallest version (FP_9_0). Library code reference names are
67 always of the largest version (in this case AR_2_0). This means that regardless
68 of the version they are added to library code, references can
"see" all names
69 that are in scope of that reference.
71 <P><span class=
"pcounter"></span>
72 Non-builtin code (client code) definitions and references are of the version it
73 was built with, as indicated by the embedding host. There is no version
74 information encoded into the ABC format to indicate its API version. In fact, it
75 is up to the host embedding to allocate version numbers and compatibility
76 relations between those versions, and to provide that information to AvmCore on
79 <P><span class=
"pcounter"></span>
80 Only public names in AVM library code (that which is built into the AVM binary)
81 may be versioned. Furthermore, it is prohibited to version names in a private,
82 protected, internal, interface, or user defined namespace. Private
83 names are not visible outside of the builtin code and so versioning has no
84 effect on them; other names might be visible outside of the builtin code but
85 are not required by currently targeted use cases to be versioned and therefore
86 doing so unnecessarily complicates the implementation.
88 <P><span class=
"pcounter"></span>
89 We also limit the set of names being versioned to those whose namespaces have
90 one of the host defined URI strings.
94 <h3>API Version Metadata
</h3>
96 The metadata syntax for marking a builtin API with a version is this:
99 VersionMetaData := '[' 'API' '(' VersionList ')' ']'
100 VersionList := VersionList ',' Version
101 Version :=
<b>Integer
</b>
104 <P><span class=
"pcounter"></span>
105 The value of
<b><code>Integer
</code></b> is determined by the host embedding. In our running
106 example of the hypothetical FR runtime, imagine that we assign the following version numbers
107 to the various versions:
119 [TODO] ASC configuration variables are defined to alias these integers. Thus to
120 indicate that a particular method was introduced in both AR_1_0 and in FP_10_0
121 we would use the following annotation:
124 [API(AR_1_0, FP_10_0)]
128 <P><span class=
"pcounter"></span>
129 Such dual versioning occurs when a name is introduced first in the larger
130 profile and later migrated to a smaller profile.
132 <P><span class=
"pcounter"></span>
133 It is an ASC compile error to use an invalid version number in API metadata.
135 <P><span class=
"pcounter"></span>
136 [NOTE: Embedders should be aware that the ASC released with the Flex SDK is
137 configured for targeting Flash Runtime versions. This is of no consequence to
138 embedders that do not version their library code and to users in general.
139 Embedders that want to version their library code and who use ASC to compile
140 that code will either need to repurpose the Flash versions and compatibility
141 relations or update the version information compiled into ASC. See the section
142 titled 'apivergen.abc' below)
144 <h3>Invoking ASC
</h3>
146 <P><span class=
"pcounter"></span>
147 When using ASC to compile builtin code with ASC's ScriptCompiler, both the
148 '-builtin' and '-apiversioning' flags must be used. This causes names in a
149 versioned namespace to be marked either with the specified version or the
150 default version (FP_9_0, in our running example).
152 <P><span class=
"pcounter"></span>
153 When using ASC to compile client code (this includes all code that is not
154 treated as built-in code by AvmCore) as any version other than the largest
155 version (AR_2_0, in our example) it is necessary to use the '-api' flag to
156 specify the presumed version of that code. The given version is used to exclude
157 names of larger versions during ABC import. ASC does nothing version specific to
158 the generated ABCs of client code. Among other things this means that third
159 party tools that read ABC will continue to work as before.
161 <h3>Invoking AVM
</h3>
163 <P><span class=
"pcounter"></span>
164 By default AVM is built with API versioning turned off. However, for the purpose
165 of testing versioning, the macro
<code>VMCFG_TEST_API_VERSIONING
</code> can be defined to
166 enable Flash Player like versioning in AVM.
168 <P><span class=
"pcounter"></span>
169 Since client version information is not stored in ABC files, the AVM has no way
170 to know what version client code is compiled against. For this reason there is
171 a command-line flag
<code>-api
</code> which allows an api version to be specified for all
172 user ABCs. [NOTE: this flag is only available with versioning is enabled as describe
173 in the previous paragraph.]
177 <h3>AVM Mechanics
</h3>
179 <P><span class=
"pcounter"></span>
180 Namespaces control the visibility of names in AS3 and AVM. We exploit this fact
181 by adding a bit mask to namespaces to indicate the version of the associated name,
182 and a bit mask to the binding (multiname hash) table entry to indicate all versions
183 in which a name is visible. A bitwise comparision between these two bitmasks is
184 done during name lookup to determine whether or not a name is a match.
186 <P><span class=
"pcounter"></span>
187 To cause versioned bindings to be visible to code of larger (and therefore different)
188 versions, we set the version bit of each larger (compatible) version in the binding
189 table. This is true for client, as well as library, code to allow SWFs of larger
190 versions to
"see" bindings of SWFs of smaller versions.
192 <P><span class=
"pcounter"></span>
193 In order to avoid changing the ABC file format we encode the version of a namespace
194 in the URI of the namespace in the ABC file and strip it out during ABC parsing,
195 replacing it with the version bitmask in the instantiated namespace object. A version
196 mark is a single Unicode character from the Private Use Area (
0xE000 to
0xF8FF). Such
197 a mark is appended to the normal (base) URI. The unicode character used to represent
198 a version has the code point equal to the sum of
0xE000 and the version number (in
199 our example, FP_9_0 namespaces would be marked with the character whose code point
202 <P><span class=
"pcounter"></span>
203 The important differences between builtin and client code are that:
205 <li>built-ins can contain bindings of different versions, whereas client code
206 bindings within an ABC are always of the same version
207 <li>built-in versions are explicitly encoded in each trait name, whereas client
208 code versions are indicated by the host
209 <li>version numbers appear nowhere in client code. this is important because
210 because version numbers might change, intentionally or accidently, from release
211 to release causing content containing that information to be brittle
214 <P><span class=
"pcounter"></span>
215 On AVM startup, the version compatibility information is passed to AvmCore via
216 the instance method
<code>AvmCore::setAPIInfo()
</code>. (See
<code>./shell/ShellCore.cpp
</code>
219 <P><span class=
"pcounter"></span>
220 During ABC parsing, the version of the ABC being parsed is given to the parser
221 by the host embedding code. Again, builtin code is by default treated as if of
222 the largest version. The version number is cached in the PoolObject associated
223 with the ABC and versioned namespaces have their appropriate version bit set.
224 When traits (and scripts for toplevel bindings) are added to the various
225 multiname hash tables the version bit on the hash table entry is set for each
226 version that is compatible with the initial version(s). [NOTE: builtin traits
227 can be introduced in multiple versions and so the set of compatible versions
228 must be computed as the union of the compatibility sets of each of the initial
231 <P><span class=
"pcounter"></span>
232 At runtime, dynamically computed names (e.g E4X names) that have a versioned URI
233 must also be versioned. AS3 and E4X namespaces share the same syntax and values
234 in the language. For example, the expression
<code>obj.public::x
</code> is a valid
235 reference to the an xml element or an ordinary object property. Therefore it is
236 necessary (or at least expedient) to also version E4X names.
238 <P><span class=
"pcounter"></span>
239 TESTING NOTE: Create an E4X value with elements with names in a namespace with a
240 versioned URI (e.g. the empty string) and passing that value to code of a smaller
241 version. Are the elements of that value visible to the receiver code? They
242 should not be, but is that a problem in practice. At least its something to document.
243 [NOTE: this is not a problem for ordinary object properties, since such
244 properties have no namespace.]
246 <P><span class=
"pcounter"></span>
247 At runtime, the PoolObject of the executing code must be derived from the dynamic
248 scope of the callee (which is builtin code). The innermost client MethodEnv
249 contains a reference to its PoolObject (through a MethodInfo). This pool contains
250 the current API version.
252 <P><span class=
"pcounter"></span>
253 Any code that reflects on the names of a value needs to reflect only those names
254 that are of the same version as the caller (e.g. describeType).
256 <P><span class=
"pcounter"></span>
257 Any code that formats a name needs to strip off the version marker of that
258 name's namespace. (For example, E4X methods that return the URI of a qname and
259 namespace, and the Multiname format method)
261 <h3>ASC Mechanics
</h3>
263 <P><span class=
"pcounter"></span>
264 ASC only marks versioned namespaces of traits when both the '-builtin' and
265 '-apiversioning' are passed to ASC's ScriptCompiler (the driver used to compile
266 builtins in the AVM shell). All other uses of ASC will result in unversioned
267 traits. Builtin traits have names that are constant multinames. Non-builtin
268 names are single qualified name constants (as before) and so there is no need to
269 bump the abc version number.
271 <P><span class=
"pcounter"></span>
272 ASC marks all builtin trait names with a version by appending a version marker
273 to the uri of the names namespace. Definitions with no version metadata are
274 marked with the smallest version (e.g. FP_9_0=
0xE000+
660). All other definitions
275 are marked with the version(s) specified by metadata.
277 <P><span class=
"pcounter"></span>
278 ASC ABC import (AbcParser.java) ignores traits that are introduced in a larger
279 version than the version of the code being compiled. The version of the current
280 code is specified by the '-api' flag on the ASC command line. It is an error to
281 use '-api' and '-apiversioning' on the same command line. This is because
282 builtin code is always of the largest version and so the '-api' flag is
283 meaningless. [NOTE: the '-api' flag has no effect on the import of '.as' files.]
285 <P><span class=
"pcounter"></span>
286 It is an error to use an invalid version number with API metadata. The definition
287 of valid versions can be found found in the file ./shell/api-versions.xml (see
293 <h3>nativegen.py
</h3>
295 <P><span class=
"pcounter"></span>nativegen.py reads the builtin ABC files and generates thunks for native
296 methods contained within. This tool ignores duplicates that result from
297 versioning, and it strips off the version marker when generating thunk names.
299 <h3>apivergen.abc
</h3>
301 <P><span class=
"pcounter"></span>
302 Version numbers and compatibility information are specified in an domain
303 specific xml file that is processed by the utility named
<code>apivergen.abc
</code>.
304 (e.g.
<code>./shell/api-versions.xml
</code>). The XML file is used as input to
305 <code>./utils/apivergen.abc
</code> to generated equivalent C++ and Java code. Copy and/or
306 rename these files as appropriate.
308 <P><span class=
"pcounter"></span>
309 To build apivergen.abc, execute
312 java -jar asc.jar apivergen.as
315 <P><span class=
"pcounter"></span>
316 To compile api-versions.xml, execute
319 avmshell apivergen.abc -- api-versions.xml
322 <P><span class=
"pcounter"></span>
324 This will result in the files
<code>api-versions.xml.h
</code> and
<code>api-versions.xml.java
</code>
325 being generated. These files must then be copied to the appropriately named files
326 (e.g. api-versions.h and APIVersions.java) in their respective locations in the
327 avm/player and ASC workspaces. [NOTE: This process should not be automated as it
328 is performed once per release.]