Class CsvDataConnector

Namespace

Datafication.Connectors.CsvConnector

Assembly

Datafication.CsvConnector.dll

Represents a CSV data connector which is used to connect to a CSV data source and retrieve data.

public class CsvDataConnector : IDataConnector

Inheritance

object

CsvDataConnector

Implements

IDataConnector

Extension Methods

DataConnectorExtensions.AsStorageConnector(IDataConnector)

DataConnectorExtensions.GetDataBatchesAsync(IDataConnector, int)

DataConnectorExtensions.GetSourceInfoAsync(IDataConnector)

Remarks

This class provides functionalities to connect to a CSV source, parse the CSV data, and retrieve data asynchronously.

Constructors

CsvDataConnector(CsvConnectorConfiguration)

Initializes a new instance of the CsvDataConnector class.

public CsvDataConnector(CsvConnectorConfiguration configuration)

Parameters

configuration CsvConnectorConfiguration

The configuration for the CSV data connector.

Remarks

This constructor ensures that the CSV data connector is properly configured before any operations are performed.

Exceptions

ArgumentNullException

Thrown when the provided configuration is null.

ArgumentException

Thrown when the configuration validation fails.

Properties

Configuration

Gets the configuration for the CSV data connector.

public CsvConnectorConfiguration Configuration { get; }

Property Value

CsvConnectorConfiguration

Methods

GetConnectorId()

Gets the connector's unique identifier.

public string GetConnectorId()

Returns

string

The unique identifier of the connector.

Remarks

This identifier is derived from the provided CsvConnectorConfiguration.

GetDataAsync()

Asynchronously retrieves data from the CSV source specified in the configuration.

public Task<DataBlock> GetDataAsync()

Returns

Task<DataBlock>

A task that represents the asynchronous operation. The task result contains a DataBlock with the CSV data.

Remarks

This method reads the CSV data from the source specified in the configuration and converts it into a DataBlock. The CSV parsing behavior (delimiter, header row, etc.) is determined by the configuration settings.

Exceptions

Exception

Thrown when there is an error reading the CSV source or when the configuration is invalid.

GetStorageDataAsync(IStorageDataBlock, int)

Asynchronously retrieves data from the CSV source and appends it in batches to the provided storage data block.

public Task<IStorageDataBlock> GetStorageDataAsync(IStorageDataBlock target, int batchSize = 10000)

Parameters

target IStorageDataBlock

The storage data block to append batches to.

batchSize int

The maximum number of rows to accumulate before appending a batch. Defaults to 10000.

Returns

Task<IStorageDataBlock>

A task that represents the asynchronous operation. The task result contains the provided IStorageDataBlock with all data appended.

Remarks

This method mirrors the exact data processing logic from GetDataAsync() but streams data in batches to the provided storage block. The CSV parsing behavior (delimiter, header row, etc.) is determined by the configuration settings.

Exceptions

ArgumentNullException

Thrown when target is null.

Exception

Thrown when there is an error reading the CSV source or when the configuration is invalid.