Use WebGL for rendering where available

[devongovett]

Still falls back to 2d canvas for old browsers. In newer browsers supporting WebGL though, there is a good performance gain. According to a profile, YCbCrToRGBA is currently the function where the most time is spent. By moving the RGB conversion to the GPU with a WebGL shader, I see a 5-10% CPU usage decrease in Chrome and Firefox, and about 15-20% decrease in Safari with WebGL enabled. You can try it for yourself here http://devongovett.github.io/jsmpeg/test.html though the loading indicator doesn't work with WebGL so just wait for it to load.

[MichielDeMey]

+1

[phoboslab]

@devongovett better late than never! The above commit (ad6abe9) implements WebGL YCbCrToRGBA conversion based on your PR.

I was reluctant to implement your PR initially, because it had some problems (no loading screen and some minor decoding artefacts) and because the platforms that needed the speed increase the most (Android/iOS) didn't support WebGL back then anyway. Now with WebGL available basically everywhere implementing it finally makes sense!

I did a benchmark with the WebGL version and @ronag's recent performance improvements for the Bick Buck Bunny video (640x360):

http://phoboslab.org/files/jsmpeg/benchmark/

• iPhone5S, iOS8.1: 4ms per frame (previously ~13ms)
• OSX, Chrome: 1.5ms per frame (previously ~4ms)

Good stuff :)

Thank you!

[ronag]

Would be nice if you updated the benchmark to 1080p. As I guess that would be a good target to work towards? With the current performance figures that should be possible.

[phoboslab]

720p example: http://phoboslab.org/files/jsmpeg/benchmark/benchmark-720p.html (still works in realtime on my iPhone5s)

I don't think 1080p mpeg1 video is a good idea by itself: the codec doesn't handle large sizes that well and the file sizes get prohibitively huge.

[devongovett]

Awesome! 👍

[MichielDeMey]

This is amazing, thanks!

[crowjonah]

I've noticed that in browser environments supporting WebGL, it is important to have the height and width attributes of the canvas element set to the dimensions of the source video being used! In my limited tests without them (or with incorrect values), the video will only occupy a section of the canvas element, and does not respond to being dynamically resized. Having those attributes set from the get-go seems to conquer these quirks, though.

[phoboslab]

@crowjonah do you have an example?

jsmpeg should set the internal canvas width & height automatically, based on the dimensions of the video. Maybe it's broken when the decoding buffer needs some padding!?

[andyearnshaw]

Ignore my previous comment, 320x180 was the correct width, but someone here had set up a 240x135 poster on the video and that's what caused the incorrect sizing. The issue with the black borders still exists in the webgl rendered videos, however.

[andyearnshaw]

I was able to fix the problem by setting up the viewport here:

if( this.gl ) {
    this.gl.useProgram(this.program);
    this.gl.viewport(0, 0, this.width, this.height);

    // ...

[phoboslab]

Well, yeah. No wonder resizing the canvas didn't work :D
Thank you!

[crowjonah]

Good catch, @andyearnshaw! Thanks all!

[andyearnshaw]

It's no problem 😄.

It's worth noting that this PR actually makes Chrome/UIWebView on iOS 8 work at an almost watchable speed. In Chrome iOS 7.1 decoding the aforementioned 320x180 video takes ~90ms, on iOS 8 it's ~44ms (tested on iPad Mini).