Hi Stefan, I believe there is a critical difference between a stream for watching TV for example, versus A/V streams in an interactive conversation.
The term "stream" is used. The thinking (inspired by the WhatWG HTML5 draft <http://www.whatwg.org/specs/web-apps/current-work/multipage/commands.html#de... ices>) is that some kind of Stream API should represent a stream. A stream in turn is what is generated by a mic or a cam, but also what would be sent between browsers.
For interactive voice, the holy grail is 150 ms, though often this is somewhat exceeded and there are several limits defined by standards, such as max 500ms allowed. For lip synchronization, interactive video must match voice. For this reason, the term "Interactive A/V Streams" may be a more accurate definition, or something similar, but distinct from present A/V streaming. What is the sentiment on list? Thanks, Henry On 1/20/11 4:42 AM, "Stefan Håkansson LK" <stefan.lk.hakansson@ericsson.com> wrote:
Just to start the discussion, I've put together the simplest use case you can imagine (with an additional error case). Some requirements are drawn from it. The intention is to add more use cases, but before spending any more energy I though it would be good to display this to the community to start the discussion and to get feedback on whether this is a workable model or not.
The requirements are split in three categories: 1. User/privacy: the intention is to list what an end user can expect in terms of what an web application can do without user consent and so on 2. API dimension: lists what must be accessible by JS and signalled to JS via APIs 3. Functional dimension: lists functionality that must be supported by the browser (in combination with the underlying operating system, drivers and HW) even though it is not visible from the API. This list is rather short since most of the functionality is visible in the API
I can't say that I am extremely satisfied with this division. Perhaps we could come up with something better. Also, some of the requirements in the User/privacy and Functional dimensions are not drawn (e.g. ask for user consent) from this use case per se, but are instead generic requirements. Perhaps they should be handled separately?
The term "stream" is used. The thinking (inspired by the WhatWG HTML5 draft <http://www.whatwg.org/specs/web-apps/current-work/multipage/commands.html#de... ices>) is that some kind of Stream API should represent a stream. A stream in turn is what is generated by a mic or a cam, but also what would be sent between browsers. Obviously these streams would have different formats (the first being raw audio/video samples, the second encoded audio/video in RTP packets). But the Steam API is what the JS code uses to manage the stream.
OK, to the use case:
==================== 1. Simple Video Chat ==================== A simple video chat service has been developed. In the service the users are logged on to the same chat web server. The web server publishes information about user login status, pushing updates to the web apps in the browsers. By clicking on an online peer user name, a 1-1 video chat session between the two browsers is initiated. The invited peer is presented with a choice of joining or rejecting the session.
The web author developing the application has decided to display a self-view as well as the video from the remote side in rather small windows, but the user can change the display size during the session. The application also supports if a participant (for a longer or shorter time) would like to stop sending audio (but keep video) or video (keep audio) to the other peer ("mute").
Any of the two participants can at any time end the chat by clicking a button.
In this specific case two users are using lap-tops in their respective homes. They are connected to the public Internet with a desktop browser using WiFi behind NATs. One of the users has an ADSL connection to the home, and the other fiber access. Most of the time headsets are used, but not always.
1.1 Requirements ================
1.1.1 User/privacy dimension ---------------------------- ------------------------------------------------------------------------------ -- !Requirement. The user must: !Comment ! ------------------------------------------------------------------------------ -- !give explicit consent before a device ! ! !can be used to capture audio or video ! ! ------------------------------------------------------------------------------ -- !be able to in an intuitive way revoke ! ! !and change capturing permissions ! ! ------------------------------------------------------------------------------ -- !be able to easily understand that audio! ! !or video is being captured ! ! ------------------------------------------------------------------------------ -- !be informed about that an invitation to! ! !a peer video chat session has been ! ! !received ! ! ------------------------------------------------------------------------------ -- !be able to accept or reject an ! ! !invitation to a peer video chat session! ! ------------------------------------------------------------------------------ -- !be able to stop a media stream from ! ! !being transmitted ! ! ------------------------------------------------------------------------------ --
1.1.2 API dimension ------------------- ------------------------------------------------------------------------------ -- !Requirement. !Comment ! ------------------------------------------------------------------------------ -- !It must be possible to update presence !Event. Out of scope for RTC-Web? ! !info from web server and make web ! ! !application aware. ! ! ------------------------------------------------------------------------------ -- !It must be possible to propagate !Out of scope for RTC-Web? ! !intention to start a chat session from ! ! !one web app (via server), and make ! ! !receiving web application aware. ! ! !Likewise, the receiving web application! ! !must be able to propagate its accept/ ! ! !reject to the initiating web app. ! ! ------------------------------------------------------------------------------ -- !The web application must be able to use!Provided the user has given consent. ! !cams and mics as input devices. ! ! ------------------------------------------------------------------------------ -- !The web application must be able to !I.e. how they are routed. To e.g. both! !control how streams generated by input !a self-view and a peer ! !devices are used ! ! ------------------------------------------------------------------------------ -- !The web application must be able to !Use the audio and video elements? ! !control how streams are rendered and ! ! !displayed ! ! ------------------------------------------------------------------------------ -- !The web application must be able to ! ! !initiate sending of streams to a peer ! ! ------------------------------------------------------------------------------ -- !The web application must be able to !If the video is going to be displayed ! !define the media format to be used for ! in a large window, use higher bit- ! !the streams sent to a peer. !rate/resolution. Should media settings! ! !be allowed to be changed during a ! ! !session (at e.g. window resize)? ! ------------------------------------------------------------------------------ -- !The web application must be made aware !Event. ! !of whether set up of stream sending was! ! !successful or not ! ! ------------------------------------------------------------------------------ -- !The web application must be made aware !Event. To be able to (with or without ! !when a stream from a peer is received !user involvement) accept or reject, ! ! !and to connect the stream to the right! ! !display/rendering element. ! ------------------------------------------------------------------------------ -- !The web application must be made aware !Event. ! !of when a stream from a peer is not ! ! !received any more ! ! ------------------------------------------------------------------------------ -- !The web application in a session must ! ! !be able to terminate all incoming and ! ! !outgoing streams ! ! ------------------------------------------------------------------------------ --
1.1.3 Functional dimension -------------------------- ------------------------------------------------------------------------------ -- !Requirement. !Comment ! ------------------------------------------------------------------------------ -- !The browser must be able to have an !Out of scope for RTC-Web? Use WS or ! !always on connection with the web !S-SE? ! !server to be able to receive presence ! ! !updates and chat initiations ! ! ------------------------------------------------------------------------------ -- !The browser must be able to use mics ! ! !and cams as input devices ! ! ------------------------------------------------------------------------------ -- !The browser must be able to send ! ! !streams (includes the associated ! ! !processing like coding, framing, etc.) ! ! !to a peer in presence of NATs. ! ! ------------------------------------------------------------------------------ -- !The browser must be able to receive ! ! !streams (associated processing) from ! ! !peers and render them ! ! ------------------------------------------------------------------------------ -- !Streams being transmitted must be !Do not starve other traffic (e.g. on ! !subject to rate control !ADSL link) ! ------------------------------------------------------------------------------ -- !When there is both incoming and !Headsets not always used ! !outgoing audio streams, echo ! ! !cancellation must be provided to avoid ! ! !disturbing echo during conversation ! ! ------------------------------------------------------------------------------ -- !Synchronization between audio and video! ! !must be supported ! ! ------------------------------------------------------------------------------ --
2 Simple video chat with link loss ================================== The use case is identical to the previous one, but in this case the ADSL link is unreliable and goes down from time to time.
2.1 Requirements ---------------- Only the delta is documented (the other reqs remain).
2.1.1 User/privacy dimension ---------------------------- ------------------------------------------------------------------------------ -- !Requirement. !Comment ! ------------------------------------------------------------------------------ -- !The user must be informed that the ! ! !communication has ceased ! ! ------------------------------------------------------------------------------ --
2.1.2 API dimension ------------------- ------------------------------------------------------------------------------ -- !Requirement. !Comment ! ------------------------------------------------------------------------------ -- !The web application must be made aware !To be able to inform user and take ! !of that the connection with the server !action. Out of scope for RTC-Web? ! !has been dropped ! ! ------------------------------------------------------------------------------ -- !The web application must be made aware !To be able to inform user and take ! !of when streams from a peer are no !action (one of the peers still has ! !longer received !connection with the server) ! ------------------------------------------------------------------------------ --
2.1.3 Functional dimension -------------------------- ------------------------------------------------------------------------------ -- !Requirement. !Comment ! ------------------------------------------------------------------------------ -- !The browser must detect when no streams! ! !are received from a peer ! ! ------------------------------------------------------------------------------ --
//Stefan _______________________________________________ dispatch mailing list dispatch@ietf.org https://www.ietf.org/mailman/listinfo/dispatch