Personal tools

Dynamic Libraries

From OpenLaszlo

(Redirected from Dynamic libraries)

Contents

Overview

A dynamic library is a portion of a program that can be loaded after the main application execution file has loaded. The goal for dynamic libraries is to reduce the initial download size of a Laszlo application. This is required because initial download size (including all requests that precede initial user interaction) is used to evaluate web applications. Reducing the initial download size has the additional benefits of reducing the the time to first interaction (although there are other ways of doing this), and of reducing bandwidth costs in the case where an optional feature of the application is not used.

Some non-goals for the initial implementation are shared libraries (where the same dynamically loaded library execution file can be be shared among multiple applications), binary distribution (where a library author can distribute a Laszlo library without distributing source code), and namespaces (where two libraries can define entities with the same name, without collision). It is intended that the initial version of dynamic libraries be forwards compatible with these features, but, in the interests of implementation time, the initial version will not implement them.

LPS 2.x, for comparison, has static libraries, which use the <library> element to define the library, and the <include> element to include it in an application (or another library). Static libraries support source code organization, team workflow, and source re-use, but they don't have any effect, versus the alternate strategy of putting everything in one big source file, on the executable file: a program that includes a static library is equivalent, to the compiler, to a program that has had the library source code textually inserted at the point of the <include> element.

Ideally, a dynamic library can contain anything that today's static library contains: class, instance, dataset, resource, and font definitions, as well as <script> blocks. In practice, the Flash implementation of Laszlo may have some limits which Laszlo inherits from the limits of the Flash file format and execution engine. We'll be exploring this over the next week; this message is intended partly to set the context for some of the discussions exploring those limits.

Requirements

  • R1. The design MUST provide a mechanism for developers to mark code that does not need to be present during the initial download.
  • R2. The design MUST provide a mechanism for developers to load this code when it does need to be present.
  • R3. The design SHOULD be compatible with the Import Statement proposal, by implementing a subset of this proposal.
  • R4. A Laszlo implementation MAY defer the execution of code marked via R1 until it is requested by the mechanism in R2.
  • R5. A Laszlo implementation for the web MAY defer the download of code marked via R1 until it is requested by the mechanism in R2.
  • R6. The Flash Laszlo implementation MUST defer the download of code marked via R1 until it is requested by the mechanism in R2, to the extent that the Flash 6 runtime provides capabilities that make this possible.

API Design

This proposal includes:

  • A means to mark part of a program as not being part of the initial download. I proposed putting this code in a <library> file and the files that it includes, and including it with an <import> statement.
  • A means to initiate loading the library. I proposed having the <import> statement define an ECMAScript object that implements a load() method.
  • A means to tell when the library has been loaded. I proposed adding an onload attribute or method to the <import> tag.

A new tag, the <import> tag, is used to include dynamic libraries. It will eventually be extended to support static libraries, with such features as namespace protection, as well..

The <import> tag must occur at the top level of an application (as the immediate child of the <canvas> or <library> document root element of a file).

The syntax of the import tag is as follows:

 <import href="mylib.lzx" stage="late" name="mylib"/>
 <import href="mylib.lzx" stage="defer" name="mylib"/>

The stage attribute is required, and may have the value "late" or "defer". A "late" library is imported once the main application file has finished loading. A "defer" library is loaded programmatically via a JavaScript method on the object bound to the variable named by the value of its "name" attribute; in this example, "mylib".

Other view attributes onload, onerror, and ontimeout are also supported:

 <import name="mylib" href="lib/foo.lzx" stage="defer" onload="doSomething()"/>

The code supplied to 'onload' will be executed once the library has loaded successfully.

It is an error for a program to contain an XML reference (TBD: define this) to a class that is defined in a late or defer library. It is unspecified whether JavaScript objects specified in a late or defer library are defined before LzLibrary.load() is called.

JavaScript API

library.load() loads a library that was imported by <import ... name="library" stage="defer"/>. If stage != 'defer' or the library was already loaded, library does nothing.

Due to the asynchronous nature of browser requests, it is possible that this call will have no immediate effect upon other properties, such as loaded, of the library object.

library.loaded is true iff the library has been loaded.

LzLibrary.load("uri", object) loads a library. The global values that the library defines are placed as attributes in object. If the function is called with only one parameter, the global values are placed in globals instead.

Implementation Design

Compiler

import elements are processed by a new class ImportCompiler, which extends ToplevelCompiler.

For illustration, assume that an instance of this class is applied to the source element <import name="mylib" href="mylib.lzx"/>.

ImportCompiler.updateSchema is the same as LibraryCompiler.updateSchema. It updates the schema of the application that contains the import with the classes defined in the imported file.

ImportCompiler.compile does two things:

  1. It compiles into the current application the equivalent of var mylib = LzLibrary("mylib.lzx");. When this is loaded, it creates an instance mylib of LzLibrary such that mylib.url == "mylib.lzx" and mylib.loaded == false.
  2. It recursively invokes the compilation manager on mylib.lzx. This creates a compiled library file that can be served in response to the runtime's request for this library. If the compilation of mylib.lzx results in an error, this is thrown again as a compilation error for the current application. If the compilation of mylib.lzx results in warnings, these warnings are appended to the list of warnings for the current application.

Runtime

Compiling a library import creates an instance of LzLibrary in the main application. LzLibrary has properties url and loaded (boolean). The library.load() method loads the library, if it has not already been loaded. It does this by requesting the URI of the library file from the server, similar to the way that Debug.eval() is done now.

Resource Requests

Resources which are specified as static (compiled into the app) will be compiled into the main app, rather than the library object file, due to restrictions on the Flash player's runtime movie loading capability.

Autoincludes

Components that are referenced in a imported library are included in the main application file, just as they are for an include. This program:

 <canvas>
   <import href="lib.lzx"/>
 </canvas>
 <library>
   <button>click</button>
 </library>

compiles as though it were written:

 <canvas>
   <include href="lz/button.lzx"/>
   <import href="lib.lzx"/>
 </canvas>
 <library>
   <button>click</button>
 </library>

This gives <import> the same semantics as <include>, with respect to autoinclude. It ought to be (relatively) easy to implement.

A different proposal would be that the autoincluded component is compiled into the library object file, not the canvas object file. I think this is riskier because the same component may be autoincluded into different libraries, and now there are more copies of the implementation in runtime. But I could be convinced otherwise.

JavaScript Target

When compiling for a non-Flash target that uses a JavaScript execution engine, the compiler can transform:

 <import name="lib" href="lib.lzx" stage="late" onload="f()">

into the equivalent of:

 <script>
   var lib = new LzLibrary;
   // code to attach the toplevel definitions in lib.lzx
   // as properties of lib; e.g.:
   lib.globalVar = 1;
   lib.globalFn = function () {}
   lib.loaded = true;
   f();
 </script>

This implements the semantics of the proposal. The stage attribute in this case is ignored.

More complicated implementations that implement the late-loading pragmatics are imaginable but not required (at least until more is known about the usage scenarios for non-Flash implementations).

Test Cases

  • errors: global imports with a name conflict; undefined dynamic import; dynamic font import (probably lots more); use name before it has been imported
  • nested, duplicate, and circular imports with various combinations of static, dynamic, local, and global
  • Compile and run a file that imports a library that defines an instance (in the test directory)
  • Compile and run a file that imports a library that defines a class (in the test directory)
  • A library that imports a library statically included in the main file
  • A library that imports another dynamic library
  • Circular imports
  • Import relative to the pathname of an included file

Attic

Dynamic Resource Notes

Consider an application that consists of two files, a canvas file and a library file. The canvas file imports the library file. (In order to show this I'll show some syntax, but this is actually a bit different from the 3.0b1 syntax --- again, I'll post on that separately.)

canvas.lzx:

 <canvas>
   <import href="library.lzx"/>
   <myclass/>
 </canvas>

library.lzx:

 <library>
   <class name="myclass" width="10" height="10" bgcolor="red"/>
 </library>

The implementation strategy is that canvas.lzx compiles into a file canvas.swf, and library.lzx compiles into library.swf. When it executes, canvas.swf loads library.swf, which executes the code therein. This has the side effect of defining the class myclass. (What's implemented now doesn't actually do this. The library has to be explicitly loaded via script, and the class therefore can't be instantiated except as script.)

This mechanism works for classes, instances, and datasets. It doesn't work for resources. Consider the single-file case. This displays a view that presents the image in logo.jpg.

 <canvas>
   <resource name="logo" src="logo.jpg"/>
   <class name="myclass" resource="logo"/>
   <myclass/>
 </canvas>

The dynamic library case ought to work the same way:

canvas.lzx:

 <canvas>
   <import href="library.lzx"/>
   <myclass/>
 </canvas>

library.lzx:

 <library>
   <resource name="logo" src="logo.jpg"/>
   <class name="myclass" resource="logo"/>
 </library>

The problem is that resources defined in one file (library.swf) don't work in another (canvas.swf).

The reason has to do with the implementation of resources in Flash files, which is something like the following. Beware: I'm not one of the Flash runtime gurus here, and I may have some of the details wrong. Someone who has worked more on the client runtime library can chime in here if this is off.

Compile-time Laszlo resources are represented as Flash characters. (Characters are also sometimes referred to definitions, symbols, or assets.) The <resource> element defines a resource named "logo", whose value is the (possibly transcoded) content of the file "logo.jpg". This resource (the name, and the content) is compiled into the canvas executable. In the Flash implementation, this is implemented as a Flash character whose name is the <resource> name.

Laszlo views are represented as JavaScript objects, with Flash movie clips as peers. (The movie clips represents the object within the Flash display list.) attachMovie is used to set a view's resource/attach a character to a clip.

Each Flash file has a character table. The character id is resolved against the table in the Flash file that contains the movie clip that the character is attached to. Movie clips are copied from clips in main.swf, so any character names are looked up in the character table in main.swf. This means a resource compiled into library.swf won't appear.

One swf can import characters that another swf has exported. The exporting swf simply lists the character ids and their export names. The importing swf names the url of the exporting swf, the export names of the exported characters, and the ids by which they're referred to locally.

Unfortunately, the swf import/export mechanism isn't useful in dynamic libraries, because the import tag must appear in the swf control tag stream (import can't be executed programatically), and it loads the url immediately.

We've been trying out other solutions. I'll send one in the next message.

Dynamic Resource Requests

[We're not doing this.]

The basic idea is to turn all resources into dynamic resource requests. No resource is compiled into a dynamic library file; instead, all resources referenced in dynamic libraries are loaded dynamically. With keepalive, this doesn't incur appreciable bandwidth or connection overhead. (There are a few more http headers.) It does create a timing window during which the dimensions of a resource, or a view that it is attached to, aren't available.

For <view src="logo.jpg"/>, this is an easy change: just turn <view src="logo.jpg"/> into <view src="http:{path}logo.jpg"/>, where {path} is the relative path from the canvas file to the library file. (I think there's already code that does this pathname adjustment for <view src="http:logo.jpg"/> in a static library source file, so the dynamic library case could just transform <view src="logo.jpg"/> to <view src="http:logo.jpg"/> and then fall through.)

For <resource name="logo" src="logo.jpg"/> in a library, generate code that adds an entry "logo" -> "{path}logo.jpg" to a table for that library. view.setResource(name) looks up whether there's an entry for "logo" for the current library, and call setSource(url) on the value of that entry if there is. Otherwise it falls through to the other cases.

For this to work, you do need to know which library a view or instance is in. LzInstantiate could set this, for example if the beginning of each library set a global to the value of the current library and the end unset it (not sure if this works if two libraries are loading at once, though).

This implementation strategy would have the advantage that it would have the side effect of implementing named dynamic resources, e.g. <resource name="logo" src="http:logo.jpg"/>. This has been a frequent request.