- Ietf-types - alvestrand.no

Media Type "text/csv" - trailing nullcols
by Yakov Shafranovich 24 Mar '05

24 Mar '05

Tak, Thanks for your comments. My intent with this draft was to document the current convention and register the MIME type. I would rather stay away from requiring or allowing features explicetly that might not be followed by all implementations. Yakov Yakov Shafranovich wrote: > From: "Tak Tang" <Tak.Tang(a)suht.swest.nhs.uk> > To: <ietf-types(a)alvestrand.no> > Date: Thu, 24 Mar 2005 11:30:43 +0000 > Subj: Media Type "text/csv" - trailing nullcols > > Hello, > > This is my first post, and I've entered the discussion late, so please > excuse me if this has already been discussed . . . > > The draft (03) does not require that each record/line have the same number > of values/commas. > I think it may be benefitial to explicitly allow csv writers to omit > trailing commas if there are no further non-null values. > > eg (all of these lines are valid) > aaa,bbb,ccc,ddd<CR-LF> > aaa,,,<CR-LF> > aaa<CR-LF> > ,bbb<CR-LF> > ,,ccc<CR-LF> > ,,,ddd<CR-LF> > > The second and third line are the same - they just have a value in the > first field. > The csv reader should assume that if there are missing columns (ie commas) > then the values are null. > Leading null values still require the commas to be included (lines 4-6). > > This is useful when > a) the dataset is sparse. > b) the dataset has new columns added - the reader can still accept old > datafiles without them being considered 'malformed'. > > Microsoft Excel and Oracle SQL*Loader can already accept data with missing > (trailing) commas. > > > Tak Tang > > > >

1 0

Media Type "text/csv" - trailing nullcols
by Tak Tang 24 Mar '05

24 Mar '05

Hello, This is my first post, and I've entered the discussion late, so please excuse me if this has already been discussed . . . The draft (03) does not require that each record/line have the same number of values/commas. I think it may be benefitial to explicitly allow csv writers to omit trailing commas if there are no further non-null values. eg (all of these lines are valid) aaa,bbb,ccc,ddd<CR-LF> aaa,,,<CR-LF> aaa<CR-LF> ,bbb<CR-LF> ,,ccc<CR-LF> ,,,ddd<CR-LF> The second and third line are the same - they just have a value in the first field. The csv reader should assume that if there are missing columns (ie commas) then the values are null. Leading null values still require the commas to be included (lines 4-6). This is useful when a) the dataset is sparse. b) the dataset has new columns added - the reader can still accept old datafiles without them being considered 'malformed'. Microsoft Excel and Oracle SQL*Loader can already accept data with missing (trailing) commas. Tak Tang D I S C L A I M E R This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. Any views or opinions presented are solely those of the author and do not necessarily represent those of the Trust unless explicitly stated otherwise. If you have received this e-mail in error please delete the e-mail and contact the Southampton University Hospitals NHS Trust Helpdesk on:- 023 80796000 The information contained in this e-mail may be subject to public disclosure under the Freedom of Information Act 2000. Unless the Information is legally exempt from disclosure, the confidentiality of this e-mail and your reply cannot be guaranteed. This footnote also confirms that this email message has been swept by MIMEsweeper for the presence of computer viruses.

1 0

Transport Area requests new/additional Media Type review for text/red
by Allison Mankin 24 Mar '05

24 Mar '05

The Transport Area requests a Media Type review for the proposed type text/red, intended for the IETF tree, as a Proposed Standard. The specification is http://www.ietf.org/internet-drafts/draft-ietf-avt-text-red-05.txt. This document had both a formal Applications AD Review and an active IETF Last Call discussion to date. During the latter, an approach of defining a "Restrictions on Usage" field for a sub-type in major type such as text that will be transported only by RTP was devised. The description of this field is found in draft-freed-media-reg-02.txt, currently in IETF Last Call itself. As the result of this, please review the following media registration for text/red instead of the one in the i-d (but read other sections in the i-d as needed): MIME media type name: text MIME subtype name: red Required parameters: rate: the RTP clock rate of the payload carried within the RTP packet. Typically, this rate is 1000, but other rates MAY be specified. This parameter MUST be set equal to the clock rate of the text payload format carried as the primary encoding. pt: a comma-separated ordered list of RTP payload types enumerating the primary, secondary, etc., in accordance with RFC 2198. Because comma is a special character, the list MUST be a quoted-string (enclosed in double quotes). For static payload types, each list element is simply the type number. For dynamic payload types, each list element is a mapping of the dynamic payload type number to an embedded MIME content- type specification for the payload format corresponding to the dynamic payload type. The format of the mapping is: dynamic-payload-type = content-type If the content-type string includes a comma, then the content- type string MUST be a quoted-string. If the content-type string does not include a comma, it MAY still be quoted. Since it is part of the list which must itself be a quoted-string, that means the quotation marks MUST be quoted with backslash quoting as specified in RFC 2045 [4]. If the content-type string itself contains a quoted-string, then the requirement for backslash quoting is recursively applied. Optional parameters: ptime, maxptime (these attributes are originally defined in RFC 2327 and RFC 3267 respectively) Restrictions on Usage: This type is defined only for transfer for RTP. It shall not be defined for a storage format. Encoding considerations: See Restrictions on Usage above; this section is included per the requirements in RFC 3555. Security considerations: Refer to section 5 of RFC XXXX. (XXXX is a reference to the RFC number to be assigned to draft-ietf-avt-text-red-0t.txt) Interoperability considerations: none Published specification: RFC XXXX. Applications which use this media type: Text streaming and conferencing tools. Additional information: none Person & email address to contact for further information: Paul E. Jones E-mail: paulej(a)packetizer.com Intended usage: COMMON Author: Paul E. Jones paulej(a)packetizer.com Change Controller: AVT Working Group delegated from the IESG Please provide any comments before 6 April 2005. Thanks, Allison

2 2

Type 'text/csv', draft-03
by Yakov Shafranovich 24 Mar '05

24 Mar '05

I posted a new version (-03) correcting the issue pointed out by Clyde Ingram. As a matter of fact while editing the draft, I noticed that the ABNF grammar did not allow commas as end of field markers for the last field and I changed the text to match. Copies of the draft can be found here (before it posts to the repository): http://www.shaftek.org/publications/drafts/mime-csv/draft-shafranovich-mime… http://www.shaftek.org/publications/drafts/mime-csv/draft-shafranovich-mime… HTML and text diff of changes: http://www.shaftek.org/publications/drafts/mime-csv/rfcdiff-02-to-03.html http://www.shaftek.org/publications/drafts/mime-csv/diff-02-to-03.txt Yakov

1 0

Re: Media Type "text/csv": new draft (-02) and Last Call
by Yakov Shafranovich 23 Mar '05

23 Mar '05

Graham Klyne wrote: > At 01:14 23/03/05 -0500, Yakov Shafranovich wrote: > >> Clyde, >> >> Thanks for pointing this out. I personally think that instead of >> making the header record mandatory which is something that most CSV >> applications do not have, I would rather take the comma out of the end >> of the record and have the last field end with a CRLF instead of an >> optional COMMA. Do you think that is a plausible solution? > > > No. Some of the Excel data I process has trailing commas. This must be > allowed. > The main intent of the document is focusing on the MIME type. The CSV common definition is an attempt at a single tight definition of the CSV format to go along with the MIME type. I don't think that every application is compliant with it but the interoperability considerations section does mention that implementors should be aware of divergent implementations. > I also don't think it's necessary to say anything (other than maybe as a > comment) about any special status for the first line: such use is > accommodated quite reasonably within the basic CSV format. > > For example, having such a line when exporting Excel as CSV depends > entirely upon how the user constructs the original spreadsheet. Column > headings are common, but not mandatory. In some cases, there may be a > more complex heading structure -- this is an application issue, not a > dataset format issue, and as such does not belong in the dataset format > specification. > Hmm... I do think that probably every single spreadsheet program I have seen uses the optional header format so it is pretty common. Since I am trying to document a common format to go along with the MIME type, something that common deserves to be documented. As for more complex cases of headers, as you have said those can be easily accomodated within the basic format while omitting the initial header line used by spreadsheets. Yakov

2 2

RE: Media Type "text/csv": new draft (-02) and Last Call
by clyde.ingram＠edl.uk.eds.com 23 Mar '05

23 Mar '05

Graham, -----Original Message----- From: Graham Klyne [mailto:GK-lists@ninebynine.org] Sent: Wednesday, March 23, 2005 9:55 AM To: Yakov Shafranovich; clyde.ingram(a)edl.uk.eds.com Cc: ietf-types(a)alvestrand.no Subject: Re: Media Type "text/csv": new draft (-02) and Last Call At 01:14 23/03/05 -0500, Yakov Shafranovich wrote: >Clyde, > >Thanks for pointing this out. I personally think that instead of making >the header record mandatory which is something that most CSV applications >do not have, I would rather take the comma out of the end of the record >and have the last field end with a CRLF instead of an optional COMMA. Do >you think that is a plausible solution? No. Some of the Excel data I process has trailing commas. This must be allowed. I also don't think it's necessary to say anything (other than maybe as a comment) about any special status for the first line: such use is accommodated quite reasonably within the basic CSV format. For example, having such a line when exporting Excel as CSV depends entirely upon how the user constructs the original spreadsheet. Column headings are common, but not mandatory. In some cases, there may be a more complex heading structure -- this is an application issue, not a dataset format issue, and as such does not belong in the dataset format specification. #g -- ------------ Please clarify whether the trailing commas that your Excel export generates are there to mark the end of the last field, or to mark the start of a last field which currently has no value. To take a concrete example, I would expect a CSV of sibling relationships in a mythical family to look like this, assuming the siblings are one brother (Bart) and 2 sisters (Lisa & Maggie): child,sisters,brothers<CR-LF> Bart,Lisa & Maggie,<CR-LF> Lisa,Maggie,Bart<CR-LF> Maggie,Lisa,Bart<CR-LF> where the trailing comma for the record of child=Bart signifies that the "brothers" field is null, so that Bart has no brothers. In my view this is a logical conclusion, and in fact stripping that one trailing comma would be an error, as that record would only have 2 fields, not 3. Would you, however, expect the CSV file to use comma as a field-terminator, rather than a field-separator, as follows?: child,sisters,brothers,<CR-LF> Bart,Lisa & Maggie,,<CR-LF> Lisa,Maggie,Bart,<CR-LF> Maggie,Lisa,Bart,<CR-LF> Note that parsers that split data records on unprotected comma would detect one field too many in this latter case. In a Comma SEPARATED Value file format, can you configure Excel to use comma as a SEPARATOR between values, rather than a TERMINATOR (at the end of values)? Regarding your remarks on the header record being "an application issue, not a dataset format issue, and as such does not belong in the dataset format specification": XML, ASN.1, and other (application-independent) data interchange formats, explicitly tag individual fields so that their type is unambiguously defined within a context. In contrast, CSV conveys no tags per field in a data record. Hence, to help with application-independent data interchange, the CSV format should convey field titles in a header record. Here is an example of lack of application-independence: if my application sends yours this CSV file: ,Bart,Lisa & Maggie<CR-LF> Bart,Lisa,Maggie<CR-LF> Bart,Maggie,Lisa<CR-LF> and your application depends on the assumption that the fields are the sequence: child sisters brothers then your application will mis-interpret the data. But if my application precedes this with a header record, like so: brothers,child,sisters<CR-LF> ,Bart,Lisa & Maggie<CR-LF> Bart,Lisa,Maggie<CR-LF> Bart,Maggie,Lisa<CR-LF> then your application can maintain independence from the change by my application, because the CSV file conveys the corresponding new field sequence (the columns "brother" and "child" have swapped). Regards, Clyde

1 0

Media Type "text/csv": new draft (-02) and Last Call
by clyde.ingram＠edl.uk.eds.com 23 Mar '05

23 Mar '05

In section "2.Definition of the CSV format", items 3 & 4 state: 3. There maybe an optional header line appearing as the first line of the file with the same format as normal record lines. This header will contain names corresponding to the fields in the file and will usually contain the same number of fields as the records in the rest of the file. For example: field_name,field_name,field_name CRLF aaa,bbb,ccc CRLF zzz,yyy,xxx CRLF 4. Within the header and each record there may be one or more fields, delimited by commas. The last field in the record may or may not be followed by a comma. For example: aaa,bbb,ccc Why would you permit the last field in the record to be followed by a comma? If a CSV record comprises: aaa,,ccc,ddd,,CRLF does it have 6 fields or 5? If comma is a field separator only, there are 6 fields: 1. aaa 2. <null> 3. ccc 4. ddd 5. <null> 6. <null> But if the comma is also a mandatory terminator for the last field (effectively the record separator becomes comma-CRLF), then there are 5 fields. In my view, permitting the last field to end with comma leads to ambiguity, and prevents an application from checking that an exact number of fields is present. The only way to guarantee the exact number of fields is then to count the fields in the header. But then your item 3 allows the header record to be omitted. Would it not be safer to make the header record mandatory? Thank-you and regards, Clyde Ingram

4 4

Media Type "text/csv": new draft (-02) and Last Call
by Yakov Shafranovich 23 Mar '05

23 Mar '05

I posted a new draft (-02) with minor ABNF corrections. The new draft can be found in the IETF repository: http://www.ietf.org/internet-drafts/draft-shafranovich-mime-csv-02.txt The HTML version, a diff of changes and all previous versions can be found here: http://www.shaftek.org/publications/drafts/mime-csv/draft-shafranovich-mime… http://www.shaftek.org/publications/drafts/mime-csv/rfcdiff-01-to-02.html http://www.shaftek.org/publications/drafts/mime-csv/ Scott Hollenbeck (APP AD) has reviewed my draft and initiated a Last Call on it which is expected to last about four weeks. Yakov

2 1

Fwd: I-D ACTION:draft-lilly-text-troff-02.txt
by Bruce Lilly 15 Mar '05

15 Mar '05

---------- Forwarded Message excerpt ---------- Subject: I-D ACTION:draft-lilly-text-troff-02.txt Date: Tue March 15 2005 10:06 From: Internet-Drafts(a)ietf.org To: i-d-announce(a)ietf.org A New Internet-Draft is available from the on-line Internet-Drafts directories. Title : Media subtype registration for media type text/troff Author(s) : B. Lilly Filename : draft-lilly-text-troff-02.txt Pages : 14 Date : 2005-3-11 A text media subtype for tagging content consisting of juxtaposed text and formatting directives as used by the troff series of programs and for conveying information about the intended processing steps necessary to produce formatted output is described. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-lilly-text-troff-02.txt To remove yourself from the I-D Announcement list, send a message to i-d-announce-request(a)ietf.org with the word unsubscribe in the body of the message. You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce to change your subscription settings. Below is the data which will enable a MIME compliant mail reader implementation to automatically retrieve the ASCII version of the Internet-Draft. ------------------------------------------------------- This is the third version of the text/troff registration (there was a second version announced shortly before IETF 62). A change history is located in Appendix B of the draft; security/privacy have been more fully addressed as a result of earlier comments. If there are no significant additional substantive comments in the next two weeks, I plan to ask the Applications Area Directors to shepherd this through the IETF/IESG process as an Informational RFC. HTML and PostScript/PDF versions formatted for ANSI A (a.k.a. "letter") and ISO A4 paper can be found at http://users.erols.com/blilly/Internet-Drafts/ ENCODING mime FILE /internet-drafts/draft-lilly-text-troff-02.txt _______________________________________________ I-D-Announce mailing list I-D-Announce(a)ietf.org https://www1.ietf.org/mailman/listinfo/i-d-announce

1 0

Transport Area requests Media Type review for audio/amr-wb+
by Allison Mankin 11 Mar '05

11 Mar '05

The Transport Area requests a Media Type review for the proposed new type w, intended for the IETF tree, as a Proposed Standard, and specified in http://www.ietf.org/internet-drafts/draft-ietf-avt-rtp-amrwbplus-06.txt (Section 7.1). Please provide any comments as soon as possible. Thanks, Allison To: ietf-types(a)alvestrand.no Subject: Transport Area requests Media Type review for Cc: hardie(a)qualcomm.com, sah(a)428cobrajet.net Fcc: +outbox Reply-To: mankin(a)psg.com -------- The Transport Area requests a Media Type review for the proposed new type w, intended for the IETF tree, as a Proposed Standard, and specified in http://www.ietf.org/internet-drafts/draft-ietf-uncomp-video-06.txt (Section ). We know of the following needed changes: 1. Author will be separated from Change Controller, and Change Controller will be modified to "AVT Working Group delegated from the IESG" Please provide any comments as soon as possible. Thanks, Allison

1 0