[Pre-RFC] Towards streaming from the foxbox

[yoric (David Teller)]

For the last week, I have been experimenting with streaming video from/through the foxbox. I don’t have anything to show for it yet, but it got me thinking about the best way to expose this through the usual REST (or native) API.

Here’s a quick braindump.

Streams are kinds of Value. For instance, an IP Camera exposing a live stream could offer a channel implementing camera/live-stream-webrtc (although for a prototype, this will more likely be camera/live-stream-html5 as this should be simpler to implement). Additional channels can be provided, e.g. camera/replay-stream-webrtc, which will take arguments to determine the start of the replay.

Stream support at least the following methods:

• start;
• pause;
• drop.

The Taxonomy Router is in charge of allocating a temporary URL for a Stream returned from an adapter and calling drop once the client is disconnected. I’m not sure how to detect disconnection yet. I’m not sure how to detect/implement a pause request, either.

Once the stream has received start and its temporary URL, it is in charge of providing a html5/webrtc video feed. Behind-the-scenes, this will be implemented through gstreamer at least for the prototype.

Any thoughts?

Cc @fabrice, @dhylands, @azasypkin, @aosmond.

edit Fixed a typo. This was live-stream, not native-stream.

[azasypkin (Oleg)]

“Temporary stream URLs” idea sounds good to me. Maybe this temporary URLs or code that generates it should accept some basic parameters like video resolution, quality, video-only/audio-only.

Also could you please clarify the following things regarding the flow:

• Are these temporary URLs generated when we call “get” for “camera/native-stream-html5” channel?

• What the difference between “drop” and “pause”? I thought that we drop a stream only once stream URL expires being in idle state for predefined period of time (or forced to expire) and pause it when all clients are disconnected, but URL is still valid/non-expired.

[yoric (David Teller)]

Are these temporary URLs generated when we call “get” for “camera/native-stream-html5” channel?

Yes, that’s the idea. Except “get” is now called “fetch”

What the difference between “drop” and “pause”? I thought that we drop a stream only once stream URL expires being in idle state for predefined period of time (or forced to expire) and pause it when all clients are disconnected, but URL is still valid/non-expired.

In my mind, pause matches the “Pause” button in the video player. The stream can remain indefinitely, at a given position. On the other hand, drop is what happens when there have been no connections to this URL for x minutes.

On the other hand, maybe “Pause” doesn’t make sense. I’m not clear about that yet.

[fabrice]

Thanks for looking at that!

I probably will sound like a broken record now, but webrtc should really be favored over http for streaming (of course there are still browsers that don’t support webrtc - I’m looking at you Safari http://caniuse.com/#search=webrtc). The bridge should not have to scale and incur costs just to relay this kind of data. One of the early designs even used webrtc datachannels for all communication with the box, only using the bridge for the initial handshake

A few other questions:

• What would the url represent in the webrtc case?
• In general we can expect the client to use a video or audio element as a basic UI block. They have controls to stop/pause/resume/seek. How will that interfere with the api proposed here? I think that we should not add burden and constraints to web developers, like preventing them from using one of the nice existing html media players.
• I’m not sure how to manage the “replay” thing in general. The backend may not always keep a buffer for instance. Should the client hint that it may need one? That’s probably ok to deal with that later though.

[yoric (David Teller)]

I probably will sound like a broken record now, but webrtc should really be favored over http for streaming (of course there are still browsers that don’t support webrtc - I’m looking at you Safari http://caniuse.com/#search=webrtc). The bridge should not have to scale and incur costs just to relay this kind of data. One of the early designs even used webrtc datachannels for all communication with the box, only using the bridge for the initial handshake

Agreed. I’m going through gstreamer atm because

1. we can afford to go through the tunnel for a first PoC;
2. once this is done, gstreamer + HTML5 can be turned into a webrtc server through either Janus or WebRTC.

What would the url represent in the webrtc case?

Frankly, I haven’t thought that far. I imagine that for WebRTC we need to replace it with some kind of UUID, with the same kind of underlying semantics start/stop, maybe pause.

In general we can expect the client to use a video or audio element as a basic UI block. They have controls to stop/pause/resume/seek. How will that interfere with the api proposed here? I think that we should not add burden and constraints to web developers, like preventing them from using one of the nice existing html media players.

I admit that I don’t know how these html media players work. If someone does, I’m interested.

I’m not sure how to manage the “replay” thing in general. The backend may not always keep a buffer for instance. Should the client hint that it may need one? That’s probably ok to deal with that later though.

I would say that a backend that stores the movie should offer the replay channel and one that doesn’t shouldn’t. In our current usecase, though, the choice is simple, since we know that we need to store the movies.