Class DataConnectorExtensions

Namespace

Datafication.Storage.Velocity.Extensions

Assembly

Datafication.Storage.Velocity.dll

Extension methods for IDataConnector to support storage operations and streaming. Provides convenience methods for upgrading existing connectors to storage-aware operations.

public static class DataConnectorExtensions

Inheritance

object

DataConnectorExtensions

Methods

AsStorageConnector(IDataConnector)

Creates a storage-aware wrapper for a regular IDataConnector. This allows existing connectors to work with the storage system.

public static IStorageConnector AsStorageConnector(this IDataConnector connector)

Parameters

connector IDataConnector

The connector to wrap.

Returns

IStorageConnector

A storage connector wrapper.

Exceptions

ArgumentNullException

Thrown when connector is null.

GetDataBatchesAsync(IDataConnector, int)

Converts a regular IDataConnector to provide basic streaming capabilities. This is a fallback implementation for connectors that don't natively support streaming.

public static IAsyncEnumerable<DataBlock> GetDataBatchesAsync(this IDataConnector connector, int batchSize = 10000)

Parameters

connector IDataConnector

The regular data connector.

batchSize int

The batch size for streaming.

Returns

IAsyncEnumerable<DataBlock>

An async enumerable of DataBlock batches.

Exceptions

ArgumentNullException

Thrown when connector is null.

ArgumentOutOfRangeException

Thrown when batchSize is less than 1.

GetSourceInfoAsync(IDataConnector)

Gets basic source information for a regular IDataConnector. This is a fallback implementation that loads the data to determine characteristics.

public static Task<DataSourceInfo> GetSourceInfoAsync(this IDataConnector connector)

Parameters

connector IDataConnector

The data connector.

Returns

Task<DataSourceInfo>

Data source information.

Exceptions

ArgumentNullException

Thrown when connector is null.

MaterializeAsync(IStorageConnector, int)

Materializes all data from a streaming operation into a single DataBlock. This is a convenience method that can be used when the full dataset is needed.

public static Task<DataBlock> MaterializeAsync(this IStorageConnector connector, int batchSize = 10000)

Parameters

connector IStorageConnector

The storage connector.

batchSize int

The batch size for streaming.

Returns

Task<DataBlock>

A single DataBlock containing all the data.

Exceptions

ArgumentNullException

Thrown when connector is null.