DataFlowComponent

Class DataFlowComponent

A base class for data flow components

Namespace: ETLBox.DataFlow
Assembly: ETLBox.dll
Syntax
    public abstract class DataFlowComponent : LoggableTask, IDataFlowComponent, ILoggableTask

Fields

WasComponentInitialized

Declaration
    protected bool WasComponentInitialized
Field Value
TypeDescription
bool

Properties

BufferCancellationSource

Declaration
    protected CancellationTokenSource BufferCancellationSource { get; set; }
Property Value
TypeDescription
CancellationTokenSource

BufferCancellationToken

A cancellation token that is linked with the underlying cancellation tokens of each buffers of a component.

Declaration
    public CancellationToken BufferCancellationToken { get; }
Property Value
TypeDescription
CancellationToken

Completion

The completion task of the component. A component is completed when all predecessors (if any) are completed and the current component has completed its buffer.

Declaration
    public Task Completion { get; }
Property Value
TypeDescription
Task

ErrorCount

Declaration
    public int ErrorCount { get; }
Property Value
TypeDescription
int

ErrorSource

The ErrorSource is the source block used for sending errors into the linked error flow.

Declaration
    public ErrorSource ErrorSource { get; set; }
Property Value
TypeDescription
ErrorSource

Exception

If a component encountered an exception or entered a fault state because another component in the data flow faulted, the thrown exception will be stored in this property.

Declaration
    public Exception Exception { get; set; }
Property Value
TypeDescription
Exception

ExecutionEndTime

Declaration
    public DateTimeOffset? ExecutionEndTime { get; set; }
Property Value
TypeDescription
DateTimeOffset?

ExecutionStartTime

Declaration
    public DateTimeOffset? ExecutionStartTime { get; set; }
Property Value
TypeDescription
DateTimeOffset?

IsReadyForProcessing

Indicates if a component was successfully initialized, linked and prepared for execution.

Declaration
    public bool IsReadyForProcessing { get; }
Property Value
TypeDescription
bool

MaxBufferSize

Each component can have one or more buffers to improve throughput and allow faster processing of data. Set this value to restrict the number of rows that can be stored in the buffer. The default value is -1 (unlimited)

Declaration
    public virtual int MaxBufferSize { get; set; }
Property Value
TypeDescription
int

OnCompletion

When a component has completed and processed all rows, the OnCompletion action is executed.

Declaration
    public Action OnCompletion { get; set; }
Property Value
TypeDescription
Action

OnException

Declaration
    public Action<Exception, string> OnException { get; set; }
Property Value
TypeDescription
Action<Exception, string>

OnInitialization

Declaration
    public Action OnInitialization { get; set; }
Property Value
TypeDescription
Action

OnProgress

This delegate is invoked each time the progress counter changes

Declaration
    public Action<int> OnProgress { get; set; }
Property Value
TypeDescription
Action<int>

Predecessors

All predecessors that are linked to this component.

Declaration
    public List<DataFlowComponent> Predecessors { get; protected set; }
Property Value
TypeDescription
List<DataFlowComponent>

ProgressCount

The amount of rows the current component has already processed.

Declaration
    public int ProgressCount { get; protected set; }
Property Value
TypeDescription
int

Successors

All successors that this component is linked to.

Declaration
    public List<DataFlowComponent> Successors { get; protected set; }
Property Value
TypeDescription
List<DataFlowComponent>

Tag

A tag that can be used to store any object in the component.

Declaration
    public object Tag { get; set; }
Property Value
TypeDescription
object

Methods

AddAutomaticVoidLinkToOtherTargets(DataFlowComponent, IDataFlowDestination)

Declaration
    protected virtual void AddAutomaticVoidLinkToOtherTargets(DataFlowComponent key, IDataFlowDestination voidDest)
Parameters
TypeNameDescription
DataFlowComponentkey
IDataFlowDestinationvoidDest

CancelBufferAndThrowOperationCanceledException()

Declaration
    protected void CancelBufferAndThrowOperationCanceledException()

CancelComponent()

Will cancel completion task for this component only. If you want to cancel the whole network, use Network.Cancel(..) instead

Declaration
    public void CancelComponent()

CheckParameter()

Declaration
    protected abstract void CheckParameter()

CleanUpOnFaulted(Exception)

Declaration
    protected virtual void CleanUpOnFaulted(Exception e = null)
Parameters
TypeNameDescription
Exceptione

CleanUpOnSuccess()

Declaration
    protected virtual void CleanUpOnSuccess()

CompleteOrFaultBufferOnPredecessorCompletion(Task)

Declaration
    protected void CompleteOrFaultBufferOnPredecessorCompletion(Task WhenAllPredecessorCompletionTasks)
Parameters
TypeNameDescription
TaskWhenAllPredecessorCompletionTasks

InitBufferObjects(CancellationToken?)

Inits the underlying TPL.Dataflow buffer objects. After this, the component is ready for linking its source or target blocks. The initialization is done automatically when a network is executed.

Declaration
    public void InitBufferObjects(CancellationToken? cancellationToken = null)
Parameters
TypeNameDescription
CancellationToken?cancellationToken

A cancellation token that is linked with the cancellation token of each buffer

InitCheckedParameter()

Declaration
    protected virtual void InitCheckedParameter()

InitComponent()

Declaration
    protected abstract void InitComponent()

InternalLinkErrorTo(IDataFlowDestination<ETLBoxError>)

Declaration
    protected IDataFlowSource<ETLBoxError> InternalLinkErrorTo(IDataFlowDestination<ETLBoxError> target)
Parameters
TypeNameDescription
IDataFlowDestination<ETLBoxError>target
Returns
TypeDescription
IDataFlowSource<ETLBoxError>

InternalLinkTo<T>(IDataFlowDestination, object, object)

Declaration
    protected IDataFlowSource<T> InternalLinkTo<T>(IDataFlowDestination target, object predicate = null, object voidPredicate = null)
Parameters
TypeNameDescription
IDataFlowDestinationtarget
objectpredicate
objectvoidPredicate
Returns
TypeDescription
IDataFlowSource<T>
Type Parameters
NameDescription
T

LinkErrorTo(IDataFlowDestination<ETLBoxError>)

If an error occurs in the component, by default the component will throw an exception and stop execution. If you use the error linking, any erroneous records will be caught and redirected.

Declaration
    public virtual IDataFlowSource<ETLBoxError> LinkErrorTo(IDataFlowDestination<ETLBoxError> target)
Parameters
TypeNameDescription
IDataFlowDestination<ETLBoxError>target

The target for erroneous rows.

Returns
TypeDescription
IDataFlowSource<ETLBoxError>

The linked component.

LogProgressBatch_AfterThrowOrRedirectError(int)

Declaration
    protected void LogProgressBatch_AfterThrowOrRedirectError(int rowsProcessed)
Parameters
TypeNameDescription
introwsProcessed

LogProgress_AfterThrowOrRedirectError()

Declaration
    protected void LogProgress_AfterThrowOrRedirectError()

LoggingFinishOnce()

Declaration
    protected void LoggingFinishOnce()

LoggingStartOnce()

Declaration
    protected void LoggingStartOnce()

PrepareParameterForCheck()

Declaration
    protected virtual void PrepareParameterForCheck()

Reset()

Declaration
    protected virtual void Reset()

ResetComponent()

Declaration
    public void ResetComponent()

SetParentComponent(DataFlowComponent)

Will set another component as a parent component - only needed for constructing subflow inside data flow components.

Declaration
    public void SetParentComponent(DataFlowComponent parent)
Parameters
TypeNameDescription
DataFlowComponentparent

ThrowErrorAndFaultNetwork(Exception, string)

Declaration
    protected void ThrowErrorAndFaultNetwork(Exception e, string erroneousData)
Parameters
TypeNameDescription
Exceptione
stringerroneousData

ThrowOrRedirectError(Exception, string, string)

Declaration
    protected void ThrowOrRedirectError(Exception e, string erroneousData, string errorContext = null)
Parameters
TypeNameDescription
Exceptione
stringerroneousData
stringerrorContext

Implements