• Getting started
  • API Documentation
  • Pricing
  • Contact
Show / Hide Table of Contents
  • ETLBox.Connection
    • AccessOdbcConnectionManager
    • AdomdConnectionManager
    • ConnectionManagerType
    • Db2ConnectionManager
    • Db2ConnectionString
    • DbConnectionManager<Connection>
    • DbConnectionString<T, TBuilder>
    • IConnectionManager
    • IDbConnectionString
    • MariaDbConnectionManager
    • MariaDbConnectionString
    • MySqlConnectionManager
    • MySqlConnectionString
    • MySqlOdbcConnectionManager
    • OdbcConnectionManager
    • OdbcConnectionString
    • OleDbConnectionManager
    • OleDbConnectionString
    • OracleConnectionManager
    • OracleConnectionString
    • OracleOdbcConnectionManager
    • PostgresConnectionManager
    • PostgresConnectionString
    • PostgresOdbcConnectionManager
    • SqlConnectionManager
    • SqlConnectionString
    • SQLiteConnectionManager
    • SQLiteConnectionString
    • SqlOdbcConnectionManager
    • SqlOleDbConnectionManager
  • ETLBox.ControlFlow
    • ControlFlow
    • ControlFlowTask
    • DbTask
    • DropTask<T>
    • IfExistsTask
    • ILoggableTask
    • ITableData
    • LoggableTask
    • ProcedureDefinition
    • ProcedureParameter
    • QueryParameter
    • TableColumn
    • TableData
    • TableData<T>
    • TableDefinition
  • ETLBox.ControlFlow.Tasks
    • CalculateDatabaseHashTask
    • CleanUpSchemaTask
    • CreateDatabaseTask
    • CreateIndexTask
    • CreateProcedureTask
    • CreateSchemaTask
    • CreateTableTask
    • CreateViewTask
    • DropDatabaseTask
    • DropIndexTask
    • DropProcedureTask
    • DropSchemaTask
    • DropTableTask
    • DropViewTask
    • GetDatabaseListTask
    • IfDatabaseExistsTask
    • IfIndexExistsTask
    • IfProcedureExistsTask
    • IfSchemaExistsTask
    • IfTableOrViewExistsTask
    • RecoveryModel
    • RowCountOptions
    • RowCountTask
    • SqlTask
    • TruncateTableTask
    • XmlaTask
  • ETLBox.DataFlow
    • AggregateColumn
    • ChangeAction
    • ColumnMap
    • CompareColumn
    • DataFlow
    • DataFlowBatchDestination<TInput>
    • DataFlowComponent
    • DataFlowDestination<TInput>
    • DataFlowExecutableSource<TOutput>
    • DataFlowJoinTarget<TInput>
    • DataFlowSource<TOutput>
    • DataFlowStreamDestination<TInput>
    • DataFlowStreamSource<TOutput>
    • DataFlowTransformation<TInput, TOutput>
    • DeleteColumn
    • DistinctColumn
    • ErrorSource
    • ETLBoxError
    • ExcelColumn
    • ExcelRange
    • GroupColumn
    • ICacheManager<TInput, TCache>
    • IDataFlowBatchDestination
    • IDataFlowBatchDestination<TInput>
    • IDataFlowComponent
    • IDataFlowDestination
    • IDataFlowDestination<TInput>
    • IDataFlowExecutableSource
    • IDataFlowExecutableSource<TOutput>
    • IDataFlowLogging
    • IDataFlowSource
    • IDataFlowSource<TOutput>
    • IDataFlowStreamDestination
    • IDataFlowStreamDestination<TInput>
    • IDataFlowStreamSource
    • IDataFlowStreamSource<TOutput>
    • IDataFlowTransformation<TInput, TOutput>
    • IDataFlowTransformation<TOutput>
    • IdColumn
    • IMergeableRow
    • JoinTarget<TInput>
    • MatchColumn
    • MemoryCache<TInput, TCache>
    • MergeableRow
    • MergeMode
    • MergeProperties
    • Network
    • ResourceType
    • RetrieveColumn
    • StreamMetaData
    • TypeInfo
    • UpdateColumn
  • ETLBox.DataFlow.Connectors
    • ConcurrentMemoryDestination
    • ConcurrentMemoryDestination<TInput>
    • CsvDestination
    • CsvDestination<TInput>
    • CsvSource
    • CsvSource<TOutput>
    • CustomBatchDestination
    • CustomBatchDestination<TInput>
    • CustomBatchSource
    • CustomBatchSource<TOutput>
    • CustomDestination
    • CustomDestination<TInput>
    • CustomSource
    • CustomSource<TOutput>
    • DbDestination
    • DbDestination<TInput>
    • DbMerge
    • DbMerge<TInput>
    • DbSource
    • DbSource<TOutput>
    • ExcelSource
    • ExcelSource<TOutput>
    • JsonDestination
    • JsonDestination<TInput>
    • JsonSource
    • JsonSource<TOutput>
    • MemoryDestination
    • MemoryDestination<TInput>
    • MemorySource
    • MemorySource<TOutput>
    • TextDestination
    • TextDestination<TInput>
    • TextSource
    • TextSource<TOutput>
    • VoidDestination
    • VoidDestination<TInput>
    • XmlDestination
    • XmlDestination<TInput>
    • XmlSource
    • XmlSource<TOutput>
  • ETLBox.DataFlow.Transformations
    • Aggregation
    • Aggregation<TInput, TOutput>
    • AggregationMethod
    • BatchTransformation
    • BatchTransformation<TInput, TOutput>
    • BatchTransformation<TInput>
    • BlockTransformation
    • BlockTransformation<TInput, TOutput>
    • BlockTransformation<TInput>
    • CachedBatchTransformation
    • CachedBatchTransformation<TInput, TOutput, TCache>
    • CachedBatchTransformation<TInput, TOutput>
    • CachedBatchTransformation<TInput>
    • CachedRowTransformation
    • CachedRowTransformation<TInput, TOutput, TCache>
    • CachedRowTransformation<TInput, TOutput>
    • CachedRowTransformation<TInput>
    • CacheMode
    • ColumnRename
    • ColumnRename<TInput>
    • CrossJoin
    • CrossJoin<TInput>
    • CrossJoin<TInput1, TInput2, TOutput>
    • CrossJoin<TInput1, TInput2, TOutput>.InMemoryDestination<TInput>
    • Distinct<TInput>
    • LookupTransformation
    • LookupTransformation<TInput, TSource>
    • LookupTransformation<TInput, TSource>.PartialDbCacheSettings
    • MergeJoin
    • MergeJoin<TInput>
    • MergeJoin<TInput1, TInput2, TOutput>
    • Multicast
    • Multicast<TInput>
    • RowDuplication
    • RowDuplication<TInput>
    • RowMultiplication
    • RowMultiplication<TInput, TOutput>
    • RowMultiplication<TInput>
    • RowTransformation
    • RowTransformation<TInput, TOutput>
    • RowTransformation<TInput>
    • Sort
    • Sort<TInput>
    • XmlSchemaValidation
    • XmlSchemaValidation<TInput>
  • ETLBox.Exceptions
    • ETLBoxException
    • ETLBoxFaultedBufferException
    • ETLBoxNotSupportedException
  • ETLBox.Helper
    • BatchLinq
    • BulkSqlGenerator<T>
    • DataTypeConverter
    • ExpandoJsonPathConverter
    • HashHelper
    • IDataTypeConverter
    • JsonPathConverter
    • JsonProperty2JsonPath
    • JsonProperty2JsonPath.MultipleResultsInPath
    • ObjectNameDescriptor
    • PropertyInfoExtension
    • SqlParser
    • StringExtension
  • ETLBox.Logging
    • AbortLoadProcessTask
    • CreateErrorTableTask
    • CreateLoadProcessTableTask
    • CreateLogTableTask
    • CustomTask
    • EndLoadProcessTask
    • ETLLogLayoutRenderer
    • GetLoadProcessAsJSONTask
    • GetLogAsJSONTask
    • LoadProcess
    • LogEntry
    • Logging
    • LogHierarchyEntry
    • LogTask
    • Package
    • ReadLoadProcessTableTask
    • ReadLogTableTask
    • ReadOptions
    • Sequence
    • Sequence<T>
    • StartLoadProcessTask

Class DataFlowSource<TOutput>

The base implementation for a source component.

Inheritance
System.Object
LoggableTask
DataFlowComponent
DataFlowSource<TOutput>
DataFlowExecutableSource<TOutput>
DataFlowTransformation<TInput, TOutput>
CrossJoin<TInput1, TInput2, TOutput>
MergeJoin<TInput1, TInput2, TOutput>
Implements
ILoggableTask
IDataFlowLogging
IDataFlowSource<TOutput>
IDataFlowSource
IDataFlowComponent
Inherited Members
DataFlowComponent.MaxBufferSize
DataFlowComponent._maxBufferSize
DataFlowComponent.Predecessors
DataFlowComponent.Successors
DataFlowComponent.Completion
DataFlowComponent.WasComponentInitialized
DataFlowComponent.ReadyForProcessing
DataFlowComponent.WasLinked
DataFlowComponent.SetParent(DataFlowComponent)
DataFlowComponent.InternalLinkTo<T>(IDataFlowDestination, Object, Object)
DataFlowComponent.LinkBuffersRecursively()
DataFlowComponent.InitBufferObjects()
DataFlowComponent.CheckParameter()
DataFlowComponent.InitComponent()
DataFlowComponent.SetCompletionTask()
DataFlowComponent.RunErrorSourceInit()
DataFlowComponent.OnCompletion
DataFlowComponent.CompleteOrFaultBufferOnPredecessorCompletion(Task)
DataFlowComponent.CleanUpOnSuccess()
DataFlowComponent.CleanUpOnFaulted(Exception)
DataFlowComponent.Exception
DataFlowComponent.ErrorSource
DataFlowComponent.HandleCanceledOrFaultedBuffer()
DataFlowComponent.InternalLinkErrorTo(IDataFlowDestination<ETLBoxError>)
DataFlowComponent.ThrowOrRedirectError(Exception, String)
DataFlowComponent.ThrowError(Exception, String)
DataFlowComponent._loggingThresholdRows
DataFlowComponent.LoggingThresholdRows
DataFlowComponent.ProgressCount
DataFlowComponent.HasLoggingThresholdRows
DataFlowComponent.ThresholdCount
DataFlowComponent.WasLoggingStarted
DataFlowComponent.WasLoggingFinished
DataFlowComponent.NLogStartOnce()
DataFlowComponent.NLogFinishOnce()
DataFlowComponent.LogProgressBatch_AfterThrowOrRedirectError(Int32)
DataFlowComponent.LogProgress_AfterThrowOrRedirectError(Boolean)
LoggableTask.TaskType
LoggableTask.TaskName
LoggableTask._disableLogging
LoggableTask.DisableLogging
LoggableTask.TaskHash
LoggableTask.CopyLogTaskProperties(ILoggableTask)
System.Object.Equals(System.Object)
System.Object.Equals(System.Object, System.Object)
System.Object.GetHashCode()
System.Object.GetType()
System.Object.MemberwiseClone()
System.Object.ReferenceEquals(System.Object, System.Object)
System.Object.ToString()
Namespace: ETLBox.DataFlow
Assembly: ETLBox.dll
Syntax
public abstract class DataFlowSource<TOutput> : DataFlowComponent, ILoggableTask, IDataFlowLogging, IDataFlowSource<TOutput>, IDataFlowSource, IDataFlowComponent
Type Parameters
Name Description
TOutput

Type of outgoing data

Properties

SourceBlock

SourceBlock from the underlying TPL.Dataflow which is used as output buffer for the component.

Declaration
public abstract ISourceBlock<TOutput> SourceBlock { get; }
Property Value
Type Description
System.Threading.Tasks.Dataflow.ISourceBlock<TOutput>

Methods

LinkErrorTo(IDataFlowDestination<ETLBoxError>)

If an error occurs in the component, by default the component will throw an exception and stop execution. If you use the error linking, any erroneous records will catched and redirected.

Declaration
public IDataFlowSource<ETLBoxError> LinkErrorTo(IDataFlowDestination<ETLBoxError> target)
Parameters
Type Name Description
IDataFlowDestination<ETLBoxError> target

The target for erroneous rows.

Returns
Type Description
IDataFlowSource<ETLBoxError>

The linked component.

LinkTo(IDataFlowDestination<TOutput>)

Links the current block to another transformation or destination. Every component should be linked to only one component without predicates If you want to link multiple targets, either use predicates or a Multicast

Declaration
public IDataFlowSource<TOutput> LinkTo(IDataFlowDestination<TOutput> target)
Parameters
Type Name Description
IDataFlowDestination<TOutput> target

Transformation or destination that the block is linked to.

Returns
Type Description
IDataFlowSource<TOutput>

The linked component.

LinkTo(IDataFlowDestination<TOutput>, Predicate<TOutput>)

Links the current block to another transformation or destination with a predicate. Every component can be linked to one or more component. If you link multiple components, provide a System.Predicate<T> that describe which row is send to which target. Make sure that all rows will be send to a target - use the VoidDestination if you want to discared rows.

Declaration
public IDataFlowSource<TOutput> LinkTo(IDataFlowDestination<TOutput> target, Predicate<TOutput> rowsToKeep)
Parameters
Type Name Description
IDataFlowDestination<TOutput> target

Transformation or destination that the block is linked to.

System.Predicate<TOutput> rowsToKeep
Returns
Type Description
IDataFlowSource<TOutput>

The linked component.

LinkTo(IDataFlowDestination<TOutput>, Predicate<TOutput>, Predicate<TOutput>)

Links the current block to another transformation or destination with a predicate for rows that you want to keep and a second predicate for rows you want to discard.

Declaration
public IDataFlowSource<TOutput> LinkTo(IDataFlowDestination<TOutput> target, Predicate<TOutput> rowsToKeep, Predicate<TOutput> rowsIntoVoid)
Parameters
Type Name Description
IDataFlowDestination<TOutput> target

Transformation or destination that the block is linked to.

System.Predicate<TOutput> rowsToKeep
System.Predicate<TOutput> rowsIntoVoid
Returns
Type Description
IDataFlowSource<TOutput>

The linked component.

LinkTo<TConvert>(IDataFlowDestination<TOutput>)

Links the current block to another transformation or destination. Every component should be linked to only one component without predicates If you want to link multiple targets, either use predicates or a Multicast

Declaration
public IDataFlowSource<TConvert> LinkTo<TConvert>(IDataFlowDestination<TOutput> target)
Parameters
Type Name Description
IDataFlowDestination<TOutput> target

Transformation or destination that the block is linked to.

Returns
Type Description
IDataFlowSource<TConvert>

The linked component.

Type Parameters
Name Description
TConvert

Will convert the output type of the linked component.

LinkTo<TConvert>(IDataFlowDestination<TOutput>, Predicate<TOutput>)

Links the current block to another transformation or destination with a predicate. Every component can be linked to one or more component. If you link multiple components, provide a System.Predicate<T> that describe which row is send to which target. Make sure that all rows will be send to a target - use the VoidDestination if you want to discared rows.

Declaration
public IDataFlowSource<TConvert> LinkTo<TConvert>(IDataFlowDestination<TOutput> target, Predicate<TOutput> rowsToKeep)
Parameters
Type Name Description
IDataFlowDestination<TOutput> target

Transformation or destination that the block is linked to.

System.Predicate<TOutput> rowsToKeep
Returns
Type Description
IDataFlowSource<TConvert>

The linked component.

Type Parameters
Name Description
TConvert

Will convert the output type of the linked component.

LinkTo<TConvert>(IDataFlowDestination<TOutput>, Predicate<TOutput>, Predicate<TOutput>)

Links the current block to another transformation or destination with a predicate for rows that you want to keep and a second predicate for rows you want to discard.

Declaration
public IDataFlowSource<TConvert> LinkTo<TConvert>(IDataFlowDestination<TOutput> target, Predicate<TOutput> rowsToKeep, Predicate<TOutput> rowsIntoVoid)
Parameters
Type Name Description
IDataFlowDestination<TOutput> target

Transformation or destination that the block is linked to.

System.Predicate<TOutput> rowsToKeep
System.Predicate<TOutput> rowsIntoVoid
Returns
Type Description
IDataFlowSource<TConvert>

The linked component.

Type Parameters
Name Description
TConvert

Will convert the output type of the linked component.

Implements

ILoggableTask
IDataFlowLogging
IDataFlowSource<TOutput>
IDataFlowSource
IDataFlowComponent
Back to top Crafted with <3 in Berlin
Contact & legal notice