Personal tools

Video

From OpenLaszlo

Contents

Contributors

Sarah Allen, Sebastian Wagner, Don Hopkins

Introduction

This proposal calls for LZX classes to support improved audio and video:

  • RTMP streaming via Flash Media Server (aka Flash Communication Server) or Red5
    • playback of recorded FLVs
    • recording from camera and microphone
    • live two-way (or multi-party) audio/video conferencing
  • FLV streaming over http

Proposal

Create a set of classes that expose logical attributes and events which make it easy to integrate video into OpenLaszlo applications, building:

  • basic classes as extensions that wrap the Flash APIs
  • higher level components, such as:
    • video player (pause, play, rewind, progress, scrubber)
    • video recording UI
    • live audio/video UI (including mic level, mute control)

API draft

The goal with the APIs is to make the simple operations very simple (few lines of LZX) offering attributes that can easily be used with constraints. More complex apps can be more complicated, yet still be able to take advantage of declarative syntax.

These APIs are now implemented, except where noted below.


Base classes

These are "extensions" which contain Flash ActionScript:

  • videoview
  • mediastream
  • mediadevice
  • camera
  • microphone
  • rtmpconnection

Components

These are "components" which written entirely in LZX and offer higher level functionality than the base classes:

  • rtmpstatus: displays a little status "light" showing red/yellow/green for disconnected/connection/connected
  • videoplayer unimplemented: video player with standard controls and a progress meter

Examples

To show a video from http server and play it automatically:

  <videoview url="http://mysite.com/myvideo.flv" autoplay="true"/>
  <videoview url="myvideo.flv" autoplay="true"/>  

From Red5 or Flash Media Server

If there is only one rtmpconnection, the video object automatically hooks up to it:

   <rtmpconnection src="rtmp://mysite.com/myapp/" autoconnect="true"/>
   <videoview url="myvideo.flv" type="rtmp" autoplay="true"/>

Add some playback controls:

  <videoview id="v" url="http://mysite.com/myvideo.flv"/>
  <button text="play" onclick="v.stream.play()"/>
  <button text="stop" onclick="v.stream.stop()"/>

Record:

   <rtmpconnection src="rtmp://mysite.com/myapp/" autoconnect="true"/>
   <videoview id="v" url="test.flv" type="rtmp">
       <camera show="true"/>
       <microphone capturing="true"/>
   </videoview>
   <view bgcolor="black" width="${v.stream.time/180*v.width}"/>
   <button text="record" onclick="v.stream.record()"/>
   <button text="stop" onclick="v.stream.stop()"/>

Record w/ more options

   <rtmpconnection src="rtmp://mysite.com/myapp/" autoconnect="true"/>
   <videoview x="10" id="v" url="test.flv" type="rtmp">
       <camera show="true" index="2"/> 
       <microphone name="mic" capturing="false"/>    
   </videoview>
   <view bgcolor="green" width="10" height="${v.mic.level/100*v.height}"/>

Record audio only:

unimplemented

   <rtmpconnection src="rtmp://mysite.com/myapp/" autoconnect="true"/>
   <stream name="rstream" url="test.flv" type="rtmp">
       <microphone/>
   </stream>

Multi-party application:

   <rtmpconnection src="rtmp://localhost/test" autoconnect="true"/>   
   <simplelayout/>
   <rtmpstatus/>
   <view layout="axis:x; inset:10; spacing:10">
       <videoview id="live" url="me" type="rtmp" oninit="this.stream.broadcast()" >
           <camera show="true"/>
       </videoview>
       <videoview id="vp" url="you" type="rtmp" oninit="this.stream.play()"/>
   </view>

Status

  • 9/8/2006 Refactored camera and microphone into shared mediadevice base class, and implemented common "allowed" permission for all devices, and proper notification in the case of multiple laszlo objects using the same underlying flash objects. Renamed "stream" to "mediastream". Numerous bug fixes, simplification and documentation. (Don Hopkins)
  • 6/28 merged video branch to trunk, note: needed to omit merge from trunk
  • 6/26 move components & extensions to proper lps directories so they can be auto-included (sallen)
  • 6/19 code review (Sarah Allen, Sebastian Wagner, A Binitie, Adam Wolff, Michael Gregor, P. Tucker Withington, Pablo Kang) Notes:
    • onlevel: can't register late (note in docs)
    • totaltime: unknown state should be null not zero
    • video play() shouldn't be a no-op, map to video.stream.play() and set playing attribute accordingly
    • if video object has null width/height, it should size itself to the incoming video
    • test, deal with stretches, clip, etc.
    • in stream _publishSetup: type is a bad variable name, since there is also a class attribute "type"
    • video/stream rtmpconnection attribute should be named "connection" so that in the future these objects may work with other protocols
    • would be nice to lose the stream.type attribute and encode the type in the URL instead
  • 6/9 most of the above proposed APIs are implemented, test files all work with FMS, red5 testing pending (red5oflaDemo still a work in progress) (sallen, swagner)
  • 4/25 branch created: http://svn.openlaszlo.org/openlaszlo/branches/video/ (jgrandy)
  • required compiler change (for creating Flash video object) is in 3.3 (sallen, hqm)

Problems and Issues

see RtmpRed5Webapp for examples

All the A/V extensions issues on JIRA:

And particulary theses issues:

 * LPP-4223 : http://jira.openlaszlo.org/jira/browse/LPP-4223  - mediastream does no stop to record/broadcast if you call stop
 * LPP-4266 : http://jira.openlaszlo.org/jira/browse/LPP-4266  - mediastream does not fire onclose event