This module provides functions and procedures for working with data formatted as comma-separated values (CSV). CSV input can be parsed into fields and CSV output can be generated. Delimiters other than the comma can be used also.
The pm_transformer module provides a simple interface to parse and generate CSV format, but the xm_csv module exports an API that can be used to solve more complex tasks involving CSV formatted data.
|It is possible to use more than one xm_csv module instance with different options in order to support different CSV formats at the same time. For this reason, functions and procedures exported by the module are public and must be referenced by the module instance name.|
This optional directive takes a single character (see below) as argument to specify the delimiter character used to separate fields. The default delimiter character is the comma (
,). Note that there is no delimiter after the last field.
This optional directive takes a single character (see below) as argument to specify the escape character used to escape special characters. The escape character is used to prefix the following characters: the escape character itself, the quote character, and the delimiter character. If EscapeControl is TRUE, the newline (
\n), carriage return (
\r), tab (
\t), and backspace (
\b) control characters are also escaped. The default escape character is the backslash character (
If this optional boolean directive is set to TRUE, control characters are also escaped. See the EscapeChar directive for details. The default is TRUE: control characters are escaped. Note that this is necessary to allow single line CSV field lists which contain line-breaks.
This optional directive specifies the list of types corresponding to the field names defined in Fields. If specified, the number of types must match the number of field names specified with Fields. If this directive is omitted, all fields will be stored as strings. This directive has no effect on the fields-to-CSV conversion.
This optional directive can take the following values:
All fields will be quoted.
Nothing will be quoted. This can be problematic if a field value (typically text that can contain any character) contains the delimiter character. Make sure that this is escaped or replaced with something else.
Note that this directive only effects CSV generation when using to_csv(). The CSV parser can automatically detect the quotation.
This directive has been deprecated in favor of QuoteMethod, which should be used instead.
If this optional boolean directive is set to TRUE, the CSV parser will fail to parse CSV lines that do not contain the required number of fields. When this is set to FALSE and the input contains fewer fields than specified in Field, the rest of the fields will be unset. The default value is FALSE.
This optional directive specifies a string which will be treated as an undefined value. This is particularly useful when parsing the W3C format where the dash (
-) marks an omitted field.
- Unquoted single character
Any printable character can be specified as an unquoted character, except for the backslash (
- Control characters
The following non-printable characters can be specified with escape sequences:
audible alert (bell)
For example, to use TAB delimiting:
- A character in single quotes
The configuration parser strips whitespace, so it is not possible to define a space as the delimiter unless it is enclosed within quotes:
Delimiter ' '
Printable characters can also be enclosed:
The backslash can be specified when enclosed within quotes:
- A character in double quotes
Double quotes can be used like single quotes:
Delimiter " "
The backslash can be specified when enclosed within double quotes:
- A hexadecimal ASCII code
Hexadecimal ASCII character codes can also be used by prepending
0x. For example, the space can be specified as:
This is equivalent to:
Delimiter " "
The following functions are exported by xm_csv.
Convert the specified fields to a single CSV formatted string.
The following procedures are exported by xm_csv.
$raw_eventfield as CSV input.
Parse the given string as CSV format.
Format the specified fields as CSV and put this into the
This example shows that the xm_csv module can not only parse and create CSV formatted input and output, but with multiple xm_csv modules it is also possible to reorder, add, remove, or modify fields before outputting to a different CSV format.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 <Extension csv1> Module xm_csv Fields $id, $name, $number FieldTypes integer, string, integer Delimiter , </Extension> <Extension csv2> Module xm_csv Fields $id, $number, $name, $date Delimiter ; </Extension> <Input in> Module im_file File "tmp/input" <Exec> csv1->parse_csv(); $date = now(); if not defined $number $number = 0; csv2->to_csv(); </Exec> </Input> <Output out> Module om_file File "tmp/output" </Output>
1, "John K.", 42 2, "Joe F.", 43
1;42;"John K.";2011-01-15 23:45:20 2;43;"Joe F.";2011-01-15 23:45:20