Struct csv_core::Reader

source ·
pub struct Reader { /* private fields */ }
Expand description

A pull based CSV reader.

This reader parses CSV data using a finite state machine. Callers can extract parsed data incrementally using one of the read methods.

Note that this CSV reader is somewhat encoding agnostic. The source data needs to be at least ASCII compatible. There is no support for specifying the full gamut of Unicode delimiters/terminators/quotes/escapes. Instead, any byte can be used, although callers probably want to stick to the ASCII subset (<= 0x7F).

Usage

A reader has two different ways to read CSV data, each with their own trade offs.

  • read_field - Copies a single CSV field into an output buffer while unescaping quotes. This is simple to use and doesn’t require storing an entire record contiguously in memory, but it is slower.
  • read_record - Copies an entire CSV record into an output buffer while unescaping quotes. The ending positions of each field are copied into an additional buffer. This is harder to use and requires larger output buffers, but it is faster than read_field since it amortizes more costs.

RFC 4180

RFC 4180 is the closest thing to a specification for CSV data. Unfortunately, CSV data that is seen in the wild can vary significantly. Often, the CSV data is outright invalid. Instead of fixing the producers of bad CSV data, we have seen fit to make consumers much more flexible in what they accept. This reader continues that tradition, and therefore, isn’t technically compliant with RFC 4180. In particular, this reader will never return an error and will always find a parse.

Here are some detailed differences from RFC 4180:

  • CRLF, LF and CR are each treated as a single record terminator by default.
  • Records are permitted to be of varying length.
  • Empty lines (that do not include other whitespace) are ignored.

Implementations§

source§

impl Reader

source

pub fn new() -> Reader

Create a new CSV reader with a default parser configuration.

source

pub fn reset(&mut self)

Reset the parser such that it behaves as if it had never been used.

This may be useful when reading CSV data in a random access pattern.

source

pub fn line(&self) -> u64

Return the current line number as measured by the number of occurrences of \n.

Line numbers starts at 1 and are reset when reset is called.

source

pub fn set_line(&mut self, line: u64)

Set the line number.

This is useful after a call to reset where the caller knows the line number from some additional context.

source

pub fn read_field( &mut self, input: &[u8], output: &mut [u8] ) -> (ReadFieldResult, usize, usize)

Parse a single CSV field in input and copy field data to output.

This routine requires a caller provided buffer of CSV data as the input and a caller provided buffer, output, in which to store field data extracted from input. The field data copied to output will have its quotes unescaped.

Calling this routine parses at most a single field and returns three values indicating the state of the parser. The first value, a ReadFieldResult, tells the caller what to do next. For example, if the entire input was read or if the output buffer was filled before a full field had been read, then ReadFieldResult::InputEmpty or ReadFieldResult::OutputFull is returned, respectively. See the documentation for ReadFieldResult for more details.

The other two values returned correspond to the number of bytes read from input and written to output, respectively.

Termination

This reader interprets an empty input buffer as an indication that there is no CSV data left to read. Namely, when the caller has exhausted all CSV data, the caller should continue to call read with an empty input buffer until ReadFieldResult::End is returned.

Errors

This CSV reader can never return an error. Instead, it prefers a parse over no parse.

source

pub fn read_record( &mut self, input: &[u8], output: &mut [u8], ends: &mut [usize] ) -> (ReadRecordResult, usize, usize, usize)

Parse a single CSV record in input and copy each field contiguously to output, with the end position of each field written to ends.

NOTE: This method is more cumbersome to use than read_field, but it can be faster since it amortizes more work.

This routine requires a caller provided buffer of CSV data as the input and two caller provided buffers to store the unescaped field data (output) and the end position of each field in the record (fields).

Calling this routine parses at most a single record and returns four values indicating the state of the parser. The first value, a ReadRecordResult, tells the caller what to do next. For example, if the entire input was read or if the output buffer was filled before a full field had been read, then ReadRecordResult::InputEmpty or ReadRecordResult::OutputFull is returned, respectively. Similarly, if the ends buffer is full, then ReadRecordResult::OutputEndsFull is returned. See the documentation for ReadRecordResult for more details.

The other three values correspond to the number of bytes read from input, the number of bytes written to output and the number of end positions written to ends, respectively.

The end positions written to ends are constructed as if there was a single contiguous buffer in memory containing the entire row, even if ReadRecordResult::OutputFull was returned in the middle of reading a row.

Termination

This reader interprets an empty input buffer as an indication that there is no CSV data left to read. Namely, when the caller has exhausted all CSV data, the caller should continue to call read with an empty input buffer until ReadRecordResult::End is returned.

Errors

This CSV reader can never return an error. Instead, it prefers a parse over no parse.

Trait Implementations§

source§

impl Clone for Reader

source§

fn clone(&self) -> Reader

Returns a copy of the value. Read more
1.0.0 · source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
source§

impl Debug for Reader

source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
source§

impl Default for Reader

source§

fn default() -> Reader

Returns the “default value” for a type. Read more

Auto Trait Implementations§

Blanket Implementations§

source§

impl<T> Any for Twhere T: 'static + ?Sized,

source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
source§

impl<T> Borrow<T> for Twhere T: ?Sized,

source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
source§

impl<T> BorrowMut<T> for Twhere T: ?Sized,

source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
source§

impl<T> From<T> for T

source§

fn from(t: T) -> T

Returns the argument unchanged.

source§

impl<T, U> Into<U> for Twhere U: From<T>,

source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

source§

impl<T> ToOwned for Twhere T: Clone,

§

type Owned = T

The resulting type after obtaining ownership.
source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
source§

impl<T, U> TryFrom<U> for Twhere U: Into<T>,

§

type Error = Infallible

The type returned in the event of a conversion error.
source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
source§

impl<T, U> TryInto<U> for Twhere U: TryFrom<T>,

§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.