This was a bad idea when it was first proposed (if I recall correctly) around ten years ago, and it's still a bad idea.

I strongly disagree.

Whenever you define an alternate representation of something, there will inevitably be skew between the original representation and the alternate representation.

This is demonstrably false. The Sieve in XML representation specified in RFC 5784 provides a counterexample - the way the format and mapping is defined, there's no way to represent anything in the XML variant that cannot be represent in the regular variant, and vice versa. In the case of Sieve, introducing the sort of skew you're worried about could only be done by violating RFC 5228 on the regular format side (by changing the core language syntax) or RFC 5784 on the XML side (by introducing an incompatible schema change). I have only done a cursory review of iCalendar in XML specification, but I believe it covers this issue adequately, and according to the document, it clearly intends to define a format that fully support round trips between the representations. If you believe it does not, it is up to you to provide specifics of how it does not, rather than asserting there's a problem without any actual evidence.

This remains true even if you define mapping rules between the two (as you do in this document). The problem with mapping rules, which I believe I pointed out around ten years ago, is that the rules are specific to a particular version of iCalendar. If either iCalendar is extended, or the XML representation is extended, there's no guidance as to how to map the extended format into the other representation.

Also demonstrably false, and once again RFC 5784 provides a counterexample. When Sieve extensions are defined the only thing that ever needs to be updated in the mapping to accomodate them is the list of controls, which is needed to map to the XML format. (And out of the many Sieve extensions that have been specified, exactly one introduced new controls, and additional extensions of this sort are very unlikely.) Morever, had this one list been viewed as a problem it could easily been eliminated, at the cost of making the XML representation a little less clear. Again, if you believe that introduction of new iCalendar properties will require updating the mapping to an unaceptable degree, it is up to you to cite specific exmaples instead of just asserting there's a problem.

In addition, defining a new calendar format harms interoperability even if you can keep the two representations in sync. The reason is that it's no longer sufficient for a calendar application to support just one representation of calendar data. In order to reliably interoperate, it must at least able to read both, and it probably needs to be able to write both. That, and when sending calendar data to other applications, either both representations must be sent, or some way of negotiating which format to use is needed, or the user must be asked to choose which format to export.

Yes, the existence of multiple formats can be an issue, but so can the inability to easily manipulate application data in popular environments using readily available tools. IMO the tradeoffs in this case are *overwhelmingly* on the side of being able to manipulate calendar data using XML tools. It should also be noted that in the case of XML, the existence of certain of those tools also lessens the difficulty of implementing conversions from XML to the original format. I know I sound like a broken record, but once again RFC 5784 provides an example of this: It includes a stylesheet to convert from the XML representation to the original format. FWIW, I think including such a conversion stylesheet would be very useful.

In summary, this is a thoroughly bad idea which can only do harm.

Again, I strongly disagree.

Please withdraw this proposal, or at least withdraw the types application until the proposal has enjoyed more review.

First of all, I'm actually kicking myself now that Sai and I didn't include a registration for application/sieve+xml in RFC 5784. If we ever do a revision I'll be sure to correct that omission. Second and far more important, the ietf-types list is for preliminary reviews only. In the case of a standards tree type like this that's being defined in an RFC, approval comes from the IESG, and that will happen as part of the review and approval process for the actual specification. It is therefore entirely appropriate for this registration to be posted to the list at this time so it can be reviewed. Now, on to the registration itself...

...

To: ietf-types@iana.org Subject: Registration of media type application/calendar+xml Type name: application Subtype name: calendar+xml Required parameters: none Optional parameters:

...

charset, method, component and optinfo as defined for the text/calendar media type

I have to question the inclusion of the charset parameter here. XML has its own internal charset label, and having this parameter opens the door to charset label silly states. Perhaps it would be better to map the text/calendar charset parameter to the internal XML label, and vide versa.

...

Encoding considerations:

...

iCalendar data is typically UTF-8 and thus the XML representation will follow that. As a result, for 7-bit transports, data in UTF-8 MUST be encoded in quoted-printable or base64.

It is customary to include a value of 7bit, 8bit, or binary here and I believe 8bit is the correct value. While it is true that XML can use UTF-16 and similar charsets that require a binary encoding label, this is a mapping of a text type, which cannot employ UTF-16.

...

Security considerations:

...

This extension does not introduce any new security concerns than those already described in iCalendar (RFC5545).

The use of XML actually raises a few additional security considerations that should be mentioned. The text in section 5 of RFC 5784 covers these.

...

Interoperability considerations: This media type provides an alternative syntax to iCalendar data based on XML. Published specification: This specification. (http://www.ietf.org/id/draft-daboo-et-al-icalendar-in-xml-06.txt) Applications which use this media type: Applications that currently make use of the text/calendar media type can use this as an alternative. Additional information: Magic number(s): None File extension(s): XML data should use "xml" as the file extension.

This is entirely your choice, but I'm not sure this recommendation is a great idea.

...

Macintosh file type code(s): None specified. Person & email address to contact for further information: Steven Lees EMail: steven.lees@microsoft.com

Although this is an individual submission, it's in the standards tree so you might consider referring this to the calendar discussion list.

...

Intended usage: COMMON Restrictions on usage: There are no restrictions on where this media type can be used. Author: Cyrus Daboo EMail: cyrus@daboo.name Mike Douglass EMail: douglm@rpi.edu Steven Lees EMail: steven.lees@microsoft.com Change controller: IETF

Ned

On Sep 9, 2010, at 11:38 PM, Ned Freed wrote:

...

...

This was a bad idea when it was first proposed (if I recall correctly) around ten years ago, and it's still a bad idea.

I strongly disagree.

...

Whenever you define an alternate representation of something, there will inevitably be skew between the original representation and the alternate representation.

This is demonstrably false. The Sieve in XML representation specified in RFC 5784 provides a counterexample - the way the format and mapping is defined, there's no way to represent anything in the XML variant that cannot be represent in the regular variant, and vice versa.

RFC 5784 might well be a valiant effort toward that. But it has only been out a few months, so I think the jury is still out as to how well it works in practice.

I'm still not hearing specifics as to how such a failure can occur, either in the Sieve XML mapping or the iCal one. Like most engineering problems, the devil is in the details, which aren't addressed in any useful fashion by sweeping pronouncements about what should or should not be attempted. A mapping to and from XML can be done well or done badly. A good mapping doesn't require schema and mapping tool changes to accomodate extensions, a bad mapping does. Let's look at a specific example. An 'fileinto "foo"' action in Sieve is mapped by RFC 5784 to XML of the form: <action name="fileinto"><str>foo</str></action> Suppose we had chosen; <fileinto>foo</fileinto> instead. Or maybe: <action name="fileinto" arg="foo"/> Both of these mappings are much simpler and easier to read, but they have a major failing: The former requires a schema change in order to add new actions, and the latter requires a schema change to accomodate any sort of extension to fileinto. By using XML elements to represent the syntactic structure of Sieve only, we create a situation where only a change to the fundamental syntax of Sieve - something the base specification specifically prohibits extensions from doing - is capable of forcing a change to the schema. iCal has a similar core syntax to Sieve that extensions have to fit into and cannot change. It follows that as long as that's all you try and represent using XML elements, extensions will not necessitate a schema change and it should be possible to write tools to perform the mapping that don't need to be updated when extensions are defined. Unfortunately, now that I've had a chance to look at draft-daboo-et-al-icalendar-in-xml-06.txt a little more, I find that it doesn't do this well. For example, instead of mapping a property like dtstamp to something like: <property name="dtstart"> <date-time>20080205T191224Z</date-time> </property> it maps it to: <dtstamp><date-time>20080205T191224Z</date-time></dtstamp> This means that additional properties will necessitate a schema update. Not good, and I believe this needs to be fixed.

I also think it helps that RFC 5784 specifically says it's not intended as an alternate storage format for Sieve. I can certainly see some utility in being able to use XML tools to manipulate things not originally written in XML.

It helps in the sense that it makes it clear who needs to support the alternate format and who doesn't. But it has no effect at all on the issue of how the mapping accomodates extensions.

But here's the acid test. If you can define a mapping from iCalendar to XML that doesn't require any string constants to describe it (other than for iCalendar keywords that imply nesting, and separators that are used in a regular fashion in iCalendar), and if you can define the inverse mapping from XML to iCalendar without naming more than a couple of specific element or parameter names - then I'll concede that the mapping will probably continue to work in the face of extensions to the iCalendar data model. Otherwise, I'm highly dubious.

I agree that this is what's needed. I don't think it's that difficult to do though. And I don't think it is difficult to evaluate whether or not that goal has been reached: All you have to do is look at the extensibility model, and make sure that anything that conforms to the extensibility model isn't going to require a schema change.

Even then, this would not address the degraded interoperability resulting from the need to have multiple formats.

It's just not this simple. The loss in interoperability may be made up for by the increase in accessibility by a different and very powerful set of tools.

...

I have only done a cursory review of iCalendar in XML specification, but I believe it covers this issue adequately, and according to the document, it clearly intends to define a format that fully support round trips between the representations. If you believe it does not, it is up to you to provide specifics of how it does not, rather than asserting there's a problem without any actual evidence.

I disagree, because there are interoperability problems resulting from the introduction of an alternate format even if the mappings remain invertable in the presence of extensions.

I'm sorry, but proof by assertion still isn't working for me here.

...

...

This remains true even if you define mapping rules between the two (as you do in this document). The problem with mapping rules, which I believe I pointed out around ten years ago, is that the rules are specific to a particular version of iCalendar. If either iCalendar is extended, or the XML representation is extended, there's no guidance as to how to map the extended format into the other representation.

Also demonstrably false, and once again RFC 5784 provides a counterexample. When Sieve extensions are defined the only thing that ever needs to be updated in the mapping to accomodate them is the list of controls, which is needed to map to the XML format.

I think you've just illustrated my point.

Tsk tsk. You're quoting me out of context, having removed the part where I said that this update could have been eliminated had we chosen to do so. In XML, there's a tradeoff between what I'll call readability (it's actually quite a bit more than that) and easy extensibility.

Though really I'm not as worried about Sieve as I doubt that there's nearly as much deployment of Sieve as there is of iCalendar.

There are demonstrably many millions of users using Sieve. The number of iCal users is probably several orders of magnitude larger, but regardless, these are both widely deployed formats. (Of course most of the people using either Sieve or iCal are unaware that's what they are using. Which is as it should be.)

I think it's far more likely that all Sieve implementations can be upgraded to support XML, if there's a desire to do that, than that all iCalendar implementations can be upgraded to support XML.

I doubt very much that all Sieve implementations will be upgraded to support both formats, nor was that ever the intent when the XML variant was defined.

I also think that Sieve<>XML is inherently saner than iCalendar<>XML because Sieve (being essentially a programming language) is already structured similarly to the way XML is structured.

Sorry, I'm not seeing the major distinction here. It's just neesting of structures in either case.

...

Again, if you believe that introduction of new iCalendar properties will require updating the mapping to an unaceptable degree, it is up to you to cite specific exmaples instead of just asserting there's a problem.

I think that the burden is on the proposers to justify producing an incompatible alternative to a existing standard, which will definitely harm interoperability.

And now you're asking the authors to prove a negative.

...

...

In addition, defining a new calendar format harms interoperability even if you can keep the two representations in sync. The reason is that it's no longer sufficient for a calendar application to support just one representation of calendar data. In order to reliably interoperate, it must at least able to read both, and it probably needs to be able to write both. That, and when sending calendar data to other applications, either both representations must be sent, or some way of negotiating which format to use is needed, or the user must be asked to choose which format to export.

Yes, the existence of multiple formats can be an issue, but so can the inability to easily manipulate application data in popular environments using readily available tools. IMO the tradeoffs in this case are *overwhelmingly* on the side of being able to manipulate calendar data using XML tools.

If people want to define mappings between iCalendar and XML just for the sake of being able to manipulate the data using XML tools, that doesn't bother me so much. It's when people want to start exchanging calendar data using XML in some cases and iCalendar in others that the interoperability problems start cropping up. (Though I have to wonder how much it really helps to be able to manipulate calendar data using XML tools, as calendar manipulations tend to be fairly specialized to calendar applications.)

Even if XML-specific tools like stylesheets prove less than useful in performing manipulations of calendar data, there's still significant benefit associated with being able to use built in parsing capbilities, espcially when those capabilites are nicely tied to automatic creation of complex data stuctures in various languages. Ned

On 10.09.2010 16:48, Ned Freed wrote:

...

... Unfortunately, now that I've had a chance to look at draft-daboo-et-al-icalendar-in-xml-06.txt a little more, I find that it doesn't do this well. For example, instead of mapping a property like dtstamp to something like:

<property name="dtstart"> <date-time>20080205T191224Z</date-time> </property>

it maps it to:

<dtstamp><date-time>20080205T191224Z</date-time></dtstamp>

This means that additional properties will necessitate a schema update. Not good, and I believe this needs to be fixed. ...

Not really. How is adding a new allowed value to an XML attribute any different from adding a new element name (assuming a schema language than can express both constraints?).

First of all, there are many situations where you don't get to pick your schema lannguage. (In fact I'd say it's the rule rather than the exception.) So even there are Schema languages that support "any element can appear here" - the fact that XML Schema doesn't allow this (xs:any has far too many restrictions placed on it to be usefullly usable) means you shouldn't be defining things this way. Second, it is of course possible to impose restrictions on the values that can appear in an attribute value (or element content for that matter). But just because you can doesn't mean you should. And these sorts of mappings are exactly the place where you shouldn't be doing this, at all, ever, because when you do you end up having to change the schema for each extension that adds properties (in the case of iCal) or actions/tests/controls (in the case of Sieve). Furthermore, in the case of Sieve at least, there are cases where it is entirely valid to specify tests and actions in a script that the implementation you're currently using doesn't support. For example: require "ihave"; if ihave "ereject" { ereject "I hate this message"; } elsif ihave "reject" { reject "I still hate this message";} else { discard; } or: require "ihave"; if ihave "x-private-extension-nobody-else-has-heard-of" { if bletch "bar" "foo" { frob :zing "foo" 1 "bar";} } If I've mapped actions and tests to elements here I'm screwed - there's no way to make this work properly in XML Schema without either adding a ton of superfluous bracketing elements all over the place (and we already have too many of these), or creating a false distinction between extensions and base elements that is, if anything, even uglier. It's also easy to show that full script validity checking here requires solving the satisfiability problem, which means its in NP. I'm pretty sure that exceeds the capabilities of most schema languages ;-) I believe similar issues exist in iCal and probably vCard. Ned

On Fri, Sep 10, 2010 at 10:48 AM, <ned+ietf@mauve.mrochek.com<ned%2Bietf@mauve.mrochek.com>

wrote:

Even if XML-specific tools like stylesheets prove less than useful in performing manipulations of calendar data, there's still significant benefit associated with being able to use built in parsing capbilities, espcially when those capabilites are nicely tied to automatic creation of complex data stuctures in various languages.

What many XML-haters do not understand is that the syntax is designed to completely automate the process of writing the parser and the backing data classes. Starting with an XML Schema definition I can generate the corresponding data structures automatically with one mouse click together with the corresponding parser/serializer calls. Starting from an EBNF description, I have to first read the description. This has already taken more time than working with the XML version would. I then have to work out if the grammar is an FSM or LR(1) or something else. When I was a grad student I used to write yacc parsers but these days I have written enough parser generators that I can actually hand code quicker than it takes me working round the peculiarities of yacc. So what takes me no time at all with XML is likely to take a couple of days and considerably more skill with EBNF. Of course this approach works best in modern languages like Java and C# but I have generated similar tools for C and I am pretty sure the same tools exist for objective C. Its going to suck somewhat if you are coding in FORTRAN or Pascal. -- Website: http://hallambaker.com/

On 9/9/2010 8:38 PM, Ned Freed wrote:

...

This was a bad idea when it was first proposed (if I recall correctly) around ten years ago, and it's still a bad idea.

I strongly disagree.

...

Whenever you define an alternate representation of something, there will inevitably be skew between the original representation and the alternate representation.

This is demonstrably false.

We need to distinguish between alternate syntactic forms, versus alternate semantic environments. Translating between versions of the former do not need to lose information. Translating between versions of the latter almost certainly do. Losing information is about differences in semantics. As I understand the calendar+xml, it is "merely" a syntactic alternative. To the extent that it requires information loss when being re-encoded, yes that should be fixed. But it's not likely to be difficult and the existence of two syntactic forms is not inherently problematic. (We have lots of examples on the net of doing this quite nicely, at different layers of Internet architecture.) As for the more abstract discussion about whether it's good or bad to have an xml version, I'll strongly suggest that it is best conducted in a real bar bof with real alcohol. (I'll be supporting its existence, FWIW.) The xml version is an important fact of life. Let's not pretend otherwise. It is not the job of the MIME registration process to make political statements that give preferential treatment to facts of life that some might like more than other facts of life... Register the damn thing. The registration form appears to satisfy registration requirements. If there are specific problems with the associated spec, pursue them independently and concretely, please. d/ -- Dave Crocker Brandenburg InternetWorking bbiw.net