Badass JavaScript

Brion Vibber has been working on ogv.js, an Ogg video player in JavaScript, supporting both audio and video. We’ve seen video codecs in JavaScript before, such as Broadway (H.264), Route9 (WebM/VP8), and more but mostly without audio support to go along with it. We have audio codecs in JS too, just not combined with video yet. That changes with ogv.js.

You can check out the demo, which runs in all modern browsers, but best in Firefox from what I’ve seen, thanks to asm.js optimizations.  I got about 23 FPS at about 40% CPU usage.  Of course, Firefox and Chrome already support Ogg video natively, so the primary targets of the project are Safari 6+ and IE 10+.  Ogv.js is an Emscripten compile of libogg, libtheora, and libvorbis.  It’s actually a fork of a project I started to bring Ogg Vorbis audio to the Aurora.js suite of audio codecs.  However, at this point, I’d be hard pressed to find any of my code in the project.

The video decoding process actually starts with a streaming HTTP implementation, which allows videos to start playing before the entire file has been downloaded.  Unfortunately, there isn’t a good streaming XHR API cross browser, so they use a combination of Microsoft's MSStreamReader API for IE 10+ (hopefully going to become standardized?), Firefox’s proprietary moz-chunked-arraybuffer, and binary strings (ouch!) for Chrome and Safari.  This allows much increased perceived performance, since decoding can happen as the file is being downloaded.

Once the video stream has been received, it is fed into the C wrapper around libogg, libtheora, and libvorbis for decoding.  Then the audio goes to the Web Audio API (which means no IE support at this point), and the video is rendered in a canvas element.  At this point, the project is just using a simple 2d canvas for rendering pixels, but they say in the readme that a WebGL implementation could be used in the future.  I’ve seen huge performance increases (up to 20%) by moving the colorspace conversion (the last step of decoding) to the graphics hardware, which can do it in parallel, so I’m looking forward to seeing this.

Parallel decoding in web workers is also being considered, since decoding both the video and audio on the main thread can make for some stutters on slower machines (read: mobile).  Support for seeking is also on the roadmap.

Video decoding in JavaScript is an interesting problem, and not something on would have expected a few years ago.  But, it seems like people keep trying it, and with each attempt, we get better.  JavaScript performance keeps increasing, especially with things like asm.js, and the new SIMD API (more on that soon!).  One by one we’re checking off the things on this list.  Who knows how practical it really is, especially since HD video or anything large still basically kills any system, and on mobile it can drain your battery really quickly.  But its a good benchmark for performance, and it’s darn cool anyway!

You can check out ogv.js on Github, and a demo as well.

P.S. I hope you like the new site design, it should be much more readable on mobile thanks to the responsive layout. Let me know what you think!