ETLBox
Title here
Summary here
BatchTransformation<TInput, TOutput>
Class BatchTransformation<TInput, TOutput>
A batch transformation will transform batches of data. The default batch size are 100000 rows. The batch transformation function allows you to process and modify each batch of data. You can use the BatchSize property to choose a smaller batch size. The batch size must always be smaller than the max buffer size. The default batch size are 1000 rows per batch. The batch transformation is a partial blocking transformation - it will always need at least enough memory to store a whole batch.
Inheritance
DataFlowSource<TOutput>
DataFlowTransformation<TInput, TOutput>
BatchTransformation<TInput, TOutput>
Implements
IDataFlowTransformation<TInput, TOutput>
IDataFlowSource<TOutput>
IDataFlowDestination<TInput>
Inherited Members
Namespace: ETLBox.DataFlow
Assembly: ETLBox.dll
Syntax
public class BatchTransformation<TInput, TOutput> : DataFlowTransformation<TInput, TOutput>, IDataFlowTransformation<TInput, TOutput>, IDataFlowSource<TOutput>, IDataFlowSource, IDataFlowDestination<TInput>, IDataFlowDestination, IDataFlowComponent, ILoggableTaskType Parameters
| Name | Description |
|---|---|
| TInput | Type of ingoing data. |
| TOutput | Type of outgoing data. |
Constructors
BatchTransformation()
Declaration
public BatchTransformation()BatchTransformation(int)
Declaration
public BatchTransformation(int batchSize)Parameters
| Type | Name | Description |
|---|---|---|
| int | batchSize | The size of each batch |
BatchTransformation(int, Func<TInput[], TOutput[]>)
Declaration
public BatchTransformation(int batchSize, Func<TInput[], TOutput[]> batchTransformationFunc)Parameters
| Type | Name | Description |
|---|---|---|
| int | batchSize | The size of each batch |
| Func<TInput[], TOutput[]> | batchTransformationFunc | Sets the BatchTransformationFunc |
Fields
WasInitActionInvoked
Declaration
protected bool WasInitActionInvokedField Value
| Type | Description |
|---|---|
| bool |
Properties
BatchSize
The size of each batch that is passed to the BatchTransformation<TInput, TOutput> Default is 1000.
Declaration
public virtual int BatchSize { get; set; }Property Value
| Type | Description |
|---|---|
| int |
BatchTransformationFunc
The transformation Func that is executed on each array of input data. It returns another array as output data - the output array can have a different length than the input array.
Declaration
public Func<TInput[], TOutput[]> BatchTransformationFunc { get; set; }Property Value
| Type | Description |
|---|---|
| Func<TInput[], TOutput[]> |
InitAction
The init action is executed shortly before the first data row is processed.
Declaration
public Action InitAction { get; set; }Property Value
| Type | Description |
|---|---|
| Action |
SourceBlock
SourceBlock from the underlying TPL.Dataflow which is used as output buffer for the component.
Declaration
public override ISourceBlock<TOutput> SourceBlock { get; }Property Value
| Type | Description |
|---|---|
| ISourceBlock<TOutput> |
Overrides
SuppressNullValueFilter
By default, all null values in the batch returned from the batch transformation func are filtered out. Set this option to true to avoid this behavior.
Declaration
public bool SuppressNullValueFilter { get; set; }Property Value
| Type | Description |
|---|---|
| bool |
TargetBlock
TargetBlock from the underlying TPL.Dataflow which is used as input buffer for the component.
Declaration
public override ITargetBlock<TInput> TargetBlock { get; }Property Value
| Type | Description |
|---|---|
| ITargetBlock<TInput> |
Overrides
Methods
CheckAndAdjustBatchSize()
Declaration
protected void CheckAndAdjustBatchSize()CheckParameter()
Declaration
protected override void CheckParameter()Overrides
CleanUpOnFaulted(Exception)
Declaration
protected override void CleanUpOnFaulted(Exception e)Parameters
| Type | Name | Description |
|---|---|---|
| Exception | e |
Overrides
CleanUpOnSuccess()
Declaration
protected override void CleanUpOnSuccess()Overrides
InitComponent()
Declaration
protected override void InitComponent()Overrides
InvokeBatchTransformationFunc(TInput[])
Declaration
protected virtual TOutput[] InvokeBatchTransformationFunc(TInput[] batch)Parameters
| Type | Name | Description |
|---|---|---|
| TInput[] | batch |
Returns
| Type | Description |
|---|---|
| TOutput[] |
InvokeInitActionOnce()
Declaration
protected virtual void InvokeInitActionOnce()Reset()
Declaration
protected override void Reset()