How to use js/moz apis to make a video of pngs?

Add-ons Development
#1

I have an API that captures all monitor content (jsctypes).
I realize this can easily be extended to screencast recording (and then next step screensharing over something).

So I was wondering how would I go about creating a video? The API gives me bitmap of all monitors at any given time. I can make a gif out of them, but what about mp4? Or youtube compatible stuff? Would be awesome if you can share some ideas/js code/or os api code to make the vids. (with jsctypes i can use os apis). or if you have c files (dll, so, dylib) to encode to video than we can tap that.

A side question: Does FPS mean frames per second? So if user wants 30fps screencast, i would take 30 screenshots per minute?

#2

whats the API that gives the bitmap of all monitors at any given time?? Could you mention it here am in need of that. Also , am in need of recording the firefox browser as a video for a specific time and store it in local. If possible could anyone suggest a solution for this too.

P.S : I need to record the browser events as a video , not as any events and repeating those events. I need to play the video when am in offline too

#3

There is no API yet for screens. You can capture a whole firefox window though using drawWindow from here: https://developer.mozilla.org/en-US/docs/Web/API/CanvasRenderingContext2D#drawWindow()

For the screens I wrote a worker that holds the function. I can hook you up with that, you will have to setup PromiseWorker, then copy paste the worker and its dependencies to your addon. Then you can use it like this:

var promise_shoot = MainWorker.post('shootAllMons', []);
promise_shoot.then(
    function(aVal) {
        console.log('Fullfilled - promise_shoot - ', aVal);
        
        // aVal is an object, with an entry per screen. each entry holds buffer, which you should put into ImageData then put to canvas OR straight right to file. entry also holds screen widht and height, and some other goodies
        
    },
    function(aReason) {
        var rejObj = {name:'promise_shoot', aReason:aReason};
        console.warn('Rejected - promise_shoot - ', rejObj);
        Services.prompt.alert(Services.wm.getMostRecentWindow('navigator:browser'), 'error', 'something went wrong internally, see browser console for details');
    }
).catch(
    function(aCaught) {
        var rejObj = {name:'promise_shoot', aCaught:aCaught};
        console.error('Caught - promise_shoot - ', rejObj);
        Services.prompt.alert(Services.wm.getMostRecentWindow('navigator:browser'), 'NativeShot - Developer Error', 'Developer did something wrong in the code, see Browser Console.');
    }
);

The worker and its dependencies are here: https://github.com/Noitidart/NativeShot/tree/master/modules

It should take 10min to setup and get to using it, if you need help jump on IRC my name is noida, im in extdev and i can help you over teamveiwer.

Heres html5 client to irc: https://client00.chat.mibbit.com/?url=irc%3A%2F%2Firc.mozilla.org%2F%23extdev

#4

To record a video of the screen you have to use WebRTC, it will prompt for permission, and it may not have as many output options. I’m sure WebRTC can also be used for screenshot. I haven’t tried it for either, please try and show code on how to do it, that would be cool.

The non-webrtc way to record screen is using jsctypes and I am looking for a collaborator, if you know C, and are willing to team up, we can knock this out with js-ctypes.