I spent the holidays thinking about various ways of dealing with the connection API problem, and I thought I might put those thoughts out for some discussion rather than trying to pick one and write it up as a draft specification. There's obviously a few models one might build upon for how to represent the peer-to-peer connections and bring them to life. The WebSockets API ( http://dev.w3.org/html5/websockets/ ) is a solution to the client-server connection problem, and the way Flash Player extended its NetConnection and NetStream objects to support peer-to-peer connectivity over RTMFP ( http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/flash/net... ) is another model. And of course draft-alvestrand-dispatch-rtcweb-datagram ( http://tools.ietf.org/html/draft-alvestrand-dispatch-rtcweb-datagram-00 ) hints at how a URL scheme might be applied to some of this problem. I believe the following items are givens, though feedback is of course welcome on these points as well: 1. When it is possible to establish a peer-to-peer connection for media, that should be supported (and is probably preferred) 2. Supporting peer-to-peer connections for media in the face of NA(P)T and firewalls will require UDP, and UDP hole-punching techniques (UDP is also preferable for delay-sensitive media) 3. Enabling a browser to send UDP datagrams to script- or server-designated IP addresses and ports is unsafe unless a handshake is used to verify that the other end is a willing participant 4. STUN Connectivity Check probes and responses (with short-term credentials) are a sufficiently strong handshake for the above (and this then suggests that ICE or a variant thereof is a good approach for the NAT traversal) 5. Given NAT and firewalls, it will also be necessary to support relayed UDP traffic; that is to say, a traffic flow which is peer-relay server-peer 6. Given NAT and firewalls, it will further be necessary to support transporting the same media traffic over a TCP connection, for cases where UDP is blocked. This could be a TCP tunnel of the UDP datagrams, WebSockets, or something else. ----- A few possible programming models to support this then present themselves, though this is by no means an exhaustive list... A. A single connection object which can magically make everything work In this scenario the API is essentially "var connection = new PeerConnection(destinationSpecifier);". The destinationSpecifier would contain enough information for the PeerConnection object's implementation to: use ICE to attempt to open a direct or relayed connection over UDP, or failing that, TCP. It would contain the information necessary to communicate with an intermediary server as the ICE negotiation proceeds as well (a "low bandwidth" signaling channel is required between a pair of peers in many cases for exchange of pertinent information before the media connection can be established). B. A single connection object which does most of the magic, but with the low-bandwidth signaling made external though events In this case, the API is the same as above, but rather than providing in the destiniationSpecifier some sort of server address for the "low bandwidth signaling" the connection object would instead fire events containing (essentially opaque to the channel, but in an agreed format) the data which needed to be transported to the other side. A corresponding API on the far end's "listening" connection object would allow for the data to be injected at the far end. The script implementer would simply need to wire the event up to an existing transport mechanism (HTTP, WebSockets). C. A connection object which does most of the magic, but with a parent object that is the connection to the signaling mechanism This is the approach Flash Player has taken. The NetConnection is opened to the rendezvous-handling server (which can also act as a media relay), NetStreams are then opened either to that relaying server or directly to other peers. These peer-to-peer NetStreams use the open NetConnection for the peer setup signaling (determining the far IP addresses, requesting NAT hole-punching, etc.) D. Multiple connection objects In this case, the API is similar to case A for the UDP peer-to-peer channel but a *different* API and connection object (perhaps an extension of the existing WebSocket API) is used for the client-server fallback cases. This requires the script to determine when a direct connection cannot be made (which sometimes cannot be determined without trying it out). Makes the programming model more complex, but avoids duplication, as existing client-server models can be used for all relayed (and server-based) media cases without there being two (or more) different ways to move real-time media from a server to a browser (and vice versa). E. Script-intensive (but flexible) components In this model, an API exists which exposes a nearly bare UDP socket, but with permissions restricted such that traffic may not be sent unless handshake probes are sent and received. Maximum flexibility, at some significant cost in script complexity (though this could be encapsulated into well-known, even open-source, libraries). An example API for this is would be: var connection = new PeerConnection(ICEusernameString, ICEpasswordString); opens the local UDP port, nothing is permitted at this point PeerConnection.testConnectivity(farSockaddrString, farUsernameString, farPasswordString); sends STUN connectivity checks PeerConnection.onConnectivityTest(receivedSockaddrString, usenameString); event that fires when a connectivity test is received with valid credentials. Browser also automatically generates the transaction response. PeerConnection.onConnectivitySuccess(receivedSockaddrString, reflexiveAddrString, usernameString); event that fires when the connectivity test transaction response arrives... Browser then adds this destination to a whitelist of addresses that media streams may be sent to PeerConnection.openMediaSession(sockaddrString) fails immediately if the address isn't on the whitelist, otherwise starts a DTLS handshake with the specified address ---- Hopefully this will stimulate discussion and/or additional ideas from other folks who are working on designing and/or implementing solutions to this problem. Matthew Kaufman matthew.kaufman@skype.net Skype: atmatthewat
Hi Matthew, good input. I agree to your baseline points. Regarding API design in general, I think that it is important that we get feedback from web developers and browser vendors, so API design related discussions should happen in the W3C space in my view. For the peer-to-peer connection API, a lot of what you bring up is related to functional split between the web app, the browser and the rendezvous/helper server. If I understand correctly, B and E would, apart from the web server only need a STUN (possibly TURN, but in the simplest case only STUN) server as support. Find and connect is handled by the application. A, C and D would need more functionality in the network (rendezvous). What has been discussed so far is more in the lines of B and E, and I think that is the path we should follow. We are doing some testing with the ConnectionPeer API, which is to my understanding close to B in your list, as descibed in the WhatWG html draft (http://www.whatwg.org/specs/web-apps/current-work/multipage/commands.html#pe...). I will come back with some experiences from that work shortly. Br, Stefan
-----Original Message----- From: rtc-web-bounces@alvestrand.no [mailto:rtc-web-bounces@alvestrand.no] On Behalf Of Matthew Kaufman Sent: den 5 januari 2011 03:01 To: RTC-Web@alvestrand.no Subject: [RTW] peer-to-peer connection API thoughts
I spent the holidays thinking about various ways of dealing with the connection API problem, and I thought I might put those thoughts out for some discussion rather than trying to pick one and write it up as a draft specification.
There's obviously a few models one might build upon for how to represent the peer-to-peer connections and bring them to life. The WebSockets API ( http://dev.w3.org/html5/websockets/ ) is a solution to the client-server connection problem, and the way Flash Player extended its NetConnection and NetStream objects to support peer-to-peer connectivity over RTMFP ( http://help.adobe.com/en_US/FlashPlatform/reference/actionscri pt/3/flash/net/NetStream.html ) is another model. And of course draft-alvestrand-dispatch-rtcweb-datagram ( http://tools.ietf.org/html/draft-alvestrand-dispatch-rtcweb-da tagram-00 ) hints at how a URL scheme might be applied to some of this problem.
I believe the following items are givens, though feedback is of course welcome on these points as well:
1. When it is possible to establish a peer-to-peer connection for media, that should be supported (and is probably preferred)
2. Supporting peer-to-peer connections for media in the face of NA(P)T and firewalls will require UDP, and UDP hole-punching techniques (UDP is also preferable for delay-sensitive media)
3. Enabling a browser to send UDP datagrams to script- or server-designated IP addresses and ports is unsafe unless a handshake is used to verify that the other end is a willing participant
4. STUN Connectivity Check probes and responses (with short-term credentials) are a sufficiently strong handshake for the above (and this then suggests that ICE or a variant thereof is a good approach for the NAT traversal)
5. Given NAT and firewalls, it will also be necessary to support relayed UDP traffic; that is to say, a traffic flow which is peer-relay server-peer
6. Given NAT and firewalls, it will further be necessary to support transporting the same media traffic over a TCP connection, for cases where UDP is blocked. This could be a TCP tunnel of the UDP datagrams, WebSockets, or something else.
-----
A few possible programming models to support this then present themselves, though this is by no means an exhaustive list...
A. A single connection object which can magically make everything work
In this scenario the API is essentially "var connection = new PeerConnection(destinationSpecifier);".
The destinationSpecifier would contain enough information for the PeerConnection object's implementation to: use ICE to attempt to open a direct or relayed connection over UDP, or failing that, TCP. It would contain the information necessary to communicate with an intermediary server as the ICE negotiation proceeds as well (a "low bandwidth" signaling channel is required between a pair of peers in many cases for exchange of pertinent information before the media connection can be established).
B. A single connection object which does most of the magic, but with the low-bandwidth signaling made external though events
In this case, the API is the same as above, but rather than providing in the destiniationSpecifier some sort of server address for the "low bandwidth signaling" the connection object would instead fire events containing (essentially opaque to the channel, but in an agreed format) the data which needed to be transported to the other side. A corresponding API on the far end's "listening" connection object would allow for the data to be injected at the far end. The script implementer would simply need to wire the event up to an existing transport mechanism (HTTP, WebSockets).
C. A connection object which does most of the magic, but with a parent object that is the connection to the signaling mechanism
This is the approach Flash Player has taken. The NetConnection is opened to the rendezvous-handling server (which can also act as a media relay), NetStreams are then opened either to that relaying server or directly to other peers. These peer-to-peer NetStreams use the open NetConnection for the peer setup signaling (determining the far IP addresses, requesting NAT hole-punching, etc.)
D. Multiple connection objects
In this case, the API is similar to case A for the UDP peer-to-peer channel but a *different* API and connection object (perhaps an extension of the existing WebSocket API) is used for the client-server fallback cases. This requires the script to determine when a direct connection cannot be made (which sometimes cannot be determined without trying it out). Makes the programming model more complex, but avoids duplication, as existing client-server models can be used for all relayed (and server-based) media cases without there being two (or more) different ways to move real-time media from a server to a browser (and vice versa).
E. Script-intensive (but flexible) components
In this model, an API exists which exposes a nearly bare UDP socket, but with permissions restricted such that traffic may not be sent unless handshake probes are sent and received. Maximum flexibility, at some significant cost in script complexity (though this could be encapsulated into well-known, even open-source, libraries).
An example API for this is would be: var connection = new PeerConnection(ICEusernameString, ICEpasswordString); opens the local UDP port, nothing is permitted at this point PeerConnection.testConnectivity(farSockaddrString, farUsernameString, farPasswordString); sends STUN connectivity checks PeerConnection.onConnectivityTest(receivedSockaddrString, usenameString); event that fires when a connectivity test is received with valid credentials. Browser also automatically generates the transaction response. PeerConnection.onConnectivitySuccess(receivedSockaddrString, reflexiveAddrString, usernameString); event that fires when the connectivity test transaction response arrives... Browser then adds this destination to a whitelist of addresses that media streams may be sent to PeerConnection.openMediaSession(sockaddrString) fails immediately if the address isn't on the whitelist, otherwise starts a DTLS handshake with the specified address
----
Hopefully this will stimulate discussion and/or additional ideas from other folks who are working on designing and/or implementing solutions to this problem.
Matthew Kaufman matthew.kaufman@skype.net Skype: atmatthewat
_______________________________________________ RTC-Web mailing list RTC-Web@alvestrand.no http://www.alvestrand.no/mailman/listinfo/rtc-web
Hi Matthew, Thanks for the detailed writeup. My previous proposal<http://rtc-web.alvestrand.com/papers/juberti-p2ptransport-api.pdf?attredirects=0> was an example of what you call model "B" - I believe this represents the most flexible option, as candidates can be exchanged through any suitable mechanism, including XmlHttpRequest, WebSockets, an existing p2p connection, IFPC, etc. It does require a bit more understanding from the programmer to figure out what to do with these callbacks, but I this concept of objects firing callbacks with blobs that need to be sent to the other side will recur in other parts of this API. One such place is in the setup of calls, where the app will need to send SDP-ish blobs to the other side over the aforementioned low-bandwidth channel. --justin On Tue, Jan 4, 2011 at 6:01 PM, Matthew Kaufman <matthew.kaufman@skype.net>wrote:
I spent the holidays thinking about various ways of dealing with the connection API problem, and I thought I might put those thoughts out for some discussion rather than trying to pick one and write it up as a draft specification.
There's obviously a few models one might build upon for how to represent the peer-to-peer connections and bring them to life. The WebSockets API ( http://dev.w3.org/html5/websockets/ ) is a solution to the client-server connection problem, and the way Flash Player extended its NetConnection and NetStream objects to support peer-to-peer connectivity over RTMFP ( http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/flash/net...) is another model. And of course draft-alvestrand-dispatch-rtcweb-datagram ( http://tools.ietf.org/html/draft-alvestrand-dispatch-rtcweb-datagram-00) hints at how a URL scheme might be applied to some of this problem.
I believe the following items are givens, though feedback is of course welcome on these points as well:
1. When it is possible to establish a peer-to-peer connection for media, that should be supported (and is probably preferred)
2. Supporting peer-to-peer connections for media in the face of NA(P)T and firewalls will require UDP, and UDP hole-punching techniques (UDP is also preferable for delay-sensitive media)
3. Enabling a browser to send UDP datagrams to script- or server-designated IP addresses and ports is unsafe unless a handshake is used to verify that the other end is a willing participant
4. STUN Connectivity Check probes and responses (with short-term credentials) are a sufficiently strong handshake for the above (and this then suggests that ICE or a variant thereof is a good approach for the NAT traversal)
5. Given NAT and firewalls, it will also be necessary to support relayed UDP traffic; that is to say, a traffic flow which is peer-relay server-peer
6. Given NAT and firewalls, it will further be necessary to support transporting the same media traffic over a TCP connection, for cases where UDP is blocked. This could be a TCP tunnel of the UDP datagrams, WebSockets, or something else.
-----
A few possible programming models to support this then present themselves, though this is by no means an exhaustive list...
A. A single connection object which can magically make everything work
In this scenario the API is essentially "var connection = new PeerConnection(destinationSpecifier);".
The destinationSpecifier would contain enough information for the PeerConnection object's implementation to: use ICE to attempt to open a direct or relayed connection over UDP, or failing that, TCP. It would contain the information necessary to communicate with an intermediary server as the ICE negotiation proceeds as well (a "low bandwidth" signaling channel is required between a pair of peers in many cases for exchange of pertinent information before the media connection can be established).
B. A single connection object which does most of the magic, but with the low-bandwidth signaling made external though events
In this case, the API is the same as above, but rather than providing in the destiniationSpecifier some sort of server address for the "low bandwidth signaling" the connection object would instead fire events containing (essentially opaque to the channel, but in an agreed format) the data which needed to be transported to the other side. A corresponding API on the far end's "listening" connection object would allow for the data to be injected at the far end. The script implementer would simply need to wire the event up to an existing transport mechanism (HTTP, WebSockets).
C. A connection object which does most of the magic, but with a parent object that is the connection to the signaling mechanism
This is the approach Flash Player has taken. The NetConnection is opened to the rendezvous-handling server (which can also act as a media relay), NetStreams are then opened either to that relaying server or directly to other peers. These peer-to-peer NetStreams use the open NetConnection for the peer setup signaling (determining the far IP addresses, requesting NAT hole-punching, etc.)
D. Multiple connection objects
In this case, the API is similar to case A for the UDP peer-to-peer channel but a *different* API and connection object (perhaps an extension of the existing WebSocket API) is used for the client-server fallback cases. This requires the script to determine when a direct connection cannot be made (which sometimes cannot be determined without trying it out). Makes the programming model more complex, but avoids duplication, as existing client-server models can be used for all relayed (and server-based) media cases without there being two (or more) different ways to move real-time media from a server to a browser (and vice versa).
E. Script-intensive (but flexible) components
In this model, an API exists which exposes a nearly bare UDP socket, but with permissions restricted such that traffic may not be sent unless handshake probes are sent and received. Maximum flexibility, at some significant cost in script complexity (though this could be encapsulated into well-known, even open-source, libraries).
An example API for this is would be: var connection = new PeerConnection(ICEusernameString, ICEpasswordString); opens the local UDP port, nothing is permitted at this point PeerConnection.testConnectivity(farSockaddrString, farUsernameString, farPasswordString); sends STUN connectivity checks PeerConnection.onConnectivityTest(receivedSockaddrString, usenameString); event that fires when a connectivity test is received with valid credentials. Browser also automatically generates the transaction response. PeerConnection.onConnectivitySuccess(receivedSockaddrString, reflexiveAddrString, usernameString); event that fires when the connectivity test transaction response arrives... Browser then adds this destination to a whitelist of addresses that media streams may be sent to PeerConnection.openMediaSession(sockaddrString) fails immediately if the address isn't on the whitelist, otherwise starts a DTLS handshake with the specified address
----
Hopefully this will stimulate discussion and/or additional ideas from other folks who are working on designing and/or implementing solutions to this problem.
Matthew Kaufman matthew.kaufman@skype.net Skype: atmatthewat
_______________________________________________ RTC-Web mailing list RTC-Web@alvestrand.no http://www.alvestrand.no/mailman/listinfo/rtc-web
Matt, thanks for these thoughts! On 01/05/11 03:01, Matthew Kaufman wrote:
I spent the holidays thinking about various ways of dealing with the connection API problem, and I thought I might put those thoughts out for some discussion rather than trying to pick one and write it up as a draft specification.
There's obviously a few models one might build upon for how to represent the peer-to-peer connections and bring them to life. The WebSockets API ( http://dev.w3.org/html5/websockets/ ) is a solution to the client-server connection problem, and the way Flash Player extended its NetConnection and NetStream objects to support peer-to-peer connectivity over RTMFP ( http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/flash/net... ) is another model. And of course draft-alvestrand-dispatch-rtcweb-datagram ( http://tools.ietf.org/html/draft-alvestrand-dispatch-rtcweb-datagram-00 ) hints at how a URL scheme might be applied to some of this problem.
I believe the following items are givens, though feedback is of course welcome on these points as well:
1. When it is possible to establish a peer-to-peer connection for media, that should be supported (and is probably preferred)
2. Supporting peer-to-peer connections for media in the face of NA(P)T and firewalls will require UDP, and UDP hole-punching techniques (UDP is also preferable for delay-sensitive media)
3. Enabling a browser to send UDP datagrams to script- or server-designated IP addresses and ports is unsafe unless a handshake is used to verify that the other end is a willing participant
4. STUN Connectivity Check probes and responses (with short-term credentials) are a sufficiently strong handshake for the above (and this then suggests that ICE or a variant thereof is a good approach for the NAT traversal)
5. Given NAT and firewalls, it will also be necessary to support relayed UDP traffic; that is to say, a traffic flow which is peer-relay server-peer
6. Given NAT and firewalls, it will further be necessary to support transporting the same media traffic over a TCP connection, for cases where UDP is blocked. This could be a TCP tunnel of the UDP datagrams, WebSockets, or something else. I like this list!
A few possible programming models to support this then present themselves, though this is by no means an exhaustive list...
A. A single connection object which can magically make everything work
In this scenario the API is essentially "var connection = new PeerConnection(destinationSpecifier);".
The destinationSpecifier would contain enough information for the PeerConnection object's implementation to: use ICE to attempt to open a direct or relayed connection over UDP, or failing that, TCP. It would contain the information necessary to communicate with an intermediary server as the ICE negotiation proceeds as well (a "low bandwidth" signaling channel is required between a pair of peers in many cases for exchange of pertinent information before the media connection can be established).
For very simple applications, this might be a sufficient API. But even if the object is magical, I think its API has to be quite a bit richer - in particular, when the state of the connection changes (failover to new connections, data doesn't flow any more, data suddenly flows a great deal faster than before) the changes have to be signalled back (using callbacks?) to the client Javascript so that appropriate user interface events can happen. When I yank out my network cable, and don't have wireless, I expect an error message saying "connection lost"; I don't expect things to just freeze. So the extended API for establishing a connection would be something like var connection = new PeerConnection(destinationSpecifier, changeCallback, deathCallback, ....)
B. A single connection object which does most of the magic, but with the low-bandwidth signaling made external though events
In this case, the API is the same as above, but rather than providing in the destiniationSpecifier some sort of server address for the "low bandwidth signaling" the connection object would instead fire events containing (essentially opaque to the channel, but in an agreed format) the data which needed to be transported to the other side. A corresponding API on the far end's "listening" connection object would allow for the data to be injected at the far end. The script implementer would simply need to wire the event up to an existing transport mechanism (HTTP, WebSockets).
I'm not familiar enough with this form of API to interpret that - what are these events, where would the events go, and who would handle them? Or are "events" Javascript terminology for what I'm used to calling "callbacks"?
C. A connection object which does most of the magic, but with a parent object that is the connection to the signaling mechanism
This is the approach Flash Player has taken. The NetConnection is opened to the rendezvous-handling server (which can also act as a media relay), NetStreams are then opened either to that relaying server or directly to other peers. These peer-to-peer NetStreams use the open NetConnection for the peer setup signaling (determining the far IP addresses, requesting NAT hole-punching, etc.)
Interesting - does this mean that one has an overarching object that can relate to multiple streams? Are there multiple streams active at the same time, and if so, what state do they share, and how are they different? Pointers to specs would be welcome!
D. Multiple connection objects
In this case, the API is similar to case A for the UDP peer-to-peer channel but a *different* API and connection object (perhaps an extension of the existing WebSocket API) is used for the client-server fallback cases. This requires the script to determine when a direct connection cannot be made (which sometimes cannot be determined without trying it out). Makes the programming model more complex, but avoids duplication, as existing client-server models can be used for all relayed (and server-based) media cases without there being two (or more) different ways to move real-time media from a server to a browser (and vice versa).
E. Script-intensive (but flexible) components
In this model, an API exists which exposes a nearly bare UDP socket, but with permissions restricted such that traffic may not be sent unless handshake probes are sent and received. Maximum flexibility, at some significant cost in script complexity (though this could be encapsulated into well-known, even open-source, libraries).
An example API for this is would be: var connection = new PeerConnection(ICEusernameString, ICEpasswordString); opens the local UDP port, nothing is permitted at this point PeerConnection.testConnectivity(farSockaddrString, farUsernameString, farPasswordString); sends STUN connectivity checks PeerConnection.onConnectivityTest(receivedSockaddrString, usenameString); event that fires when a connectivity test is received with valid credentials. Browser also automatically generates the transaction response. PeerConnection.onConnectivitySuccess(receivedSockaddrString, reflexiveAddrString, usernameString); event that fires when the connectivity test transaction response arrives... Browser then adds this destination to a whitelist of addresses that media streams may be sent to PeerConnection.openMediaSession(sockaddrString) fails immediately if the address isn't on the whitelist, otherwise starts a DTLS handshake with the specified address
No more comments on these - good starting thoughts!
On 1/7/2011 5:03 AM, Harald Alvestrand wrote:
For very simple applications, this might be a sufficient API. But even if the object is magical, I think its API has to be quite a bit richer - in particular, when the state of the connection changes (failover to new connections, data doesn't flow any more, data suddenly flows a great deal faster than before) the changes have to be signalled back (using callbacks?) to the client Javascript so that appropriate user interface events can happen.
When I yank out my network cable, and don't have wireless, I expect an error message saying "connection lost"; I don't expect things to just freeze.
Agreed. I had left out all the status events and adjustment knobs... trying to just get connection establishment happening.
So the extended API for establishing a connection would be something like
var connection = new PeerConnection(destinationSpecifier, changeCallback, deathCallback, ....)
In JavaScript you'd just have the destinationSpecifier and then assign event handlers to the "connection" instance of the PeerConnection object. So: connection.onDeath = doSomething; (or in the new way, connection.addEventListener('death', doSomething, false); )
B. A single connection object which does most of the magic, but with the low-bandwidth signaling made external though events
In this case, the API is the same as above, but rather than providing in the destiniationSpecifier some sort of server address for the "low bandwidth signaling" the connection object would instead fire events containing (essentially opaque to the channel, but in an agreed format) the data which needed to be transported to the other side. A corresponding API on the far end's "listening" connection object would allow for the data to be injected at the far end. The script implementer would simply need to wire the event up to an existing transport mechanism (HTTP, WebSockets).
I'm not familiar enough with this form of API to interpret that - what are these events, where would the events go, and who would handle them? Or are "events" Javascript terminology for what I'm used to calling "callbacks"?
Pretty much, yes. See above. So you'd have something like: function sendMessageToOtherEnd(message) { ... use an existing WebSocket to your signaling server to send the message ... } connection.onMessageToOtherEnd = sendMessageToOtherEnd;
C. A connection object which does most of the magic, but with a parent object that is the connection to the signaling mechanism
This is the approach Flash Player has taken. The NetConnection is opened to the rendezvous-handling server (which can also act as a media relay), NetStreams are then opened either to that relaying server or directly to other peers. These peer-to-peer NetStreams use the open NetConnection for the peer setup signaling (determining the far IP addresses, requesting NAT hole-punching, etc.)
Interesting - does this mean that one has an overarching object that can relate to multiple streams? Are there multiple streams active at the same time, and if so, what state do they share, and how are they different?
Yes. There's a "NetConnection" which represents the connection to the server, and then zero or more NetStreams, each of which is a unidirectional flow of media. Some of the NetStreams may represent peer-to-peer streams, so either "published locally for consumption by peer" or "played from <remote peer-ID>".
Pointers to specs would be welcome!
http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/flash/net... and http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/flash/net... (The same API is also used by "RTMFP Groups", which is a p2p overlay that includes application-level multicast, so you can also publish/play NetStreams over the multicast overlay.)
No more comments on these - good starting thoughts!
I'm glad to see that we're starting to make progress on multiple fronts here. Shouldn't be long before we can put together some draft specs and start refining the details. Matthew Kaufman
Hi Matthew, all, before drafting all the API possibilities in detail, perhaps we should discuss what should be standardized in terms of signaling mechanisms? As I read your proposal, APIs A, C and D all rely on some sort of connection control signaling happening under the API (i.e being part of the implementation in the browser). B and E in contrast leaves this to the application developer. Any means can be used to exchange the required information, and the standard would be silent on what mean to use. In http://datatracker.ietf.org/doc/draft-alvestrand-dispatch-rtcweb-protocols/?... it is stated that connection control signaling is left to (web app) implementors. My impression was that there is a consensus on not standardizing signaling mechanisms (at least not in the first phase), but to focus on enabling the browser to set up a connection to a peer and to transmit and recieve media streams. (I see a need, for cases where you like to enable interoperable applications, to agree on signaling formats - but this is another thing, it is not part of the browser implementation but rather e.g. a common JS library) Maybe I'm getting something wrong here, please correct me in that case. Also, IMHO, if we are to standardize signaling mechanisms as part of the implementation in the browser, SIP/SDP would be a natural starting point. //Stefan
-----Original Message----- From: rtc-web-bounces@alvestrand.no [mailto:rtc-web-bounces@alvestrand.no] On Behalf Of Matthew Kaufman Sent: den 8 januari 2011 05:30 To: Harald Alvestrand Cc: RTC-Web@alvestrand.no Subject: Re: [RTW] peer-to-peer connection API thoughts
On 1/7/2011 5:03 AM, Harald Alvestrand wrote:
For very simple applications, this might be a sufficient
API. But even
if the object is magical, I think its API has to be quite a bit richer - in particular, when the state of the connection changes (failover to new connections, data doesn't flow any more, data suddenly flows a great deal faster than before) the changes have to be signalled back (using callbacks?) to the client Javascript so that appropriate user interface events can happen.
When I yank out my network cable, and don't have wireless, I expect an error message saying "connection lost"; I don't expect things to just freeze.
Agreed. I had left out all the status events and adjustment knobs... trying to just get connection establishment happening.
So the extended API for establishing a connection would be something like
var connection = new PeerConnection(destinationSpecifier, changeCallback, deathCallback, ....)
In JavaScript you'd just have the destinationSpecifier and then assign event handlers to the "connection" instance of the PeerConnection object. So:
connection.onDeath = doSomething;
(or in the new way, connection.addEventListener('death', doSomething, false); )
B. A single connection object which does most of the
magic, but with
the low-bandwidth signaling made external though events
In this case, the API is the same as above, but rather than providing in the destiniationSpecifier some sort of server address for the "low bandwidth signaling" the connection object would instead fire events containing (essentially opaque to the channel, but in an agreed format) the data which needed to be transported to the other side. A corresponding API on the far end's "listening" connection object would allow for the data to be injected at the far end. The script implementer would simply need to wire the event up to an existing transport mechanism (HTTP, WebSockets). I'm not familiar enough with this form of API to interpret that - what are these events, where would the events go, and who would handle them? Or are "events" Javascript terminology for what I'm used to calling "callbacks"?
Pretty much, yes. See above.
So you'd have something like:
function sendMessageToOtherEnd(message) { ... use an existing WebSocket to your signaling server to send the message ... }
connection.onMessageToOtherEnd = sendMessageToOtherEnd;
C. A connection object which does most of the magic, but with a parent object that is the connection to the signaling mechanism
This is the approach Flash Player has taken. The NetConnection is opened to the rendezvous-handling server (which can also act as a media relay), NetStreams are then opened either to that relaying server or directly to other peers. These peer-to-peer
NetStreams use
the open NetConnection for the peer setup signaling (determining the far IP addresses, requesting NAT hole-punching, etc.) Interesting - does this mean that one has an overarching object that can relate to multiple streams? Are there multiple streams active at the same time, and if so, what state do they share, and how are they different?
Yes. There's a "NetConnection" which represents the connection to the server, and then zero or more NetStreams, each of which is a unidirectional flow of media. Some of the NetStreams may represent peer-to-peer streams, so either "published locally for consumption by peer" or "played from <remote peer-ID>".
Pointers to specs would be welcome!
http://help.adobe.com/en_US/FlashPlatform/reference/actionscri pt/3/flash/net/NetConnection.html
and
http://help.adobe.com/en_US/FlashPlatform/reference/actionscri pt/3/flash/net/NetStream.html
(The same API is also used by "RTMFP Groups", which is a p2p overlay that includes application-level multicast, so you can also publish/play NetStreams over the multicast overlay.)
No more comments on these - good starting thoughts!
I'm glad to see that we're starting to make progress on multiple fronts here. Shouldn't be long before we can put together some draft specs and start refining the details.
Matthew Kaufman _______________________________________________ RTC-Web mailing list RTC-Web@alvestrand.no http://www.alvestrand.no/mailman/listinfo/rtc-web
participants (4)
-
Harald Alvestrand -
Justin Uberti -
Matthew Kaufman -
Stefan Håkansson LK