• Getting started
  • API Documentation
  • Pricing
  • Contact
Show / Hide Table of Contents
  • ETLBox.Connection
    • AccessOdbcConnectionManager
    • AdomdConnectionManager
    • ConnectionManagerType
    • Db2ConnectionManager
    • Db2ConnectionString
    • DbConnectionManager<Connection>
    • DbConnectionString<T, TBuilder>
    • IConnectionManager
    • IDbConnectionString
    • MariaDbConnectionManager
    • MariaDbConnectionString
    • MySqlConnectionManager
    • MySqlConnectionString
    • MySqlOdbcConnectionManager
    • OdbcConnectionManager
    • OdbcConnectionString
    • OleDbConnectionManager
    • OleDbConnectionString
    • OracleConnectionManager
    • OracleConnectionString
    • OracleOdbcConnectionManager
    • PostgresConnectionManager
    • PostgresConnectionString
    • PostgresOdbcConnectionManager
    • SqlConnectionManager
    • SqlConnectionString
    • SQLiteConnectionManager
    • SQLiteConnectionString
    • SqlOdbcConnectionManager
    • SqlOleDbConnectionManager
  • ETLBox.ControlFlow
    • ControlFlow
    • ControlFlowTask
    • DbTask
    • DropTask<T>
    • IfExistsTask
    • ILoggableTask
    • ITableData
    • LoggableTask
    • ProcedureDefinition
    • ProcedureParameter
    • QueryParameter
    • TableColumn
    • TableData
    • TableData<T>
    • TableDefinition
  • ETLBox.ControlFlow.Tasks
    • CalculateDatabaseHashTask
    • CleanUpSchemaTask
    • CreateDatabaseTask
    • CreateIndexTask
    • CreateProcedureTask
    • CreateSchemaTask
    • CreateTableTask
    • CreateViewTask
    • DropDatabaseTask
    • DropIndexTask
    • DropProcedureTask
    • DropSchemaTask
    • DropTableTask
    • DropViewTask
    • GetDatabaseListTask
    • IfDatabaseExistsTask
    • IfIndexExistsTask
    • IfProcedureExistsTask
    • IfSchemaExistsTask
    • IfTableOrViewExistsTask
    • RecoveryModel
    • RowCountOptions
    • RowCountTask
    • SqlTask
    • TruncateTableTask
    • XmlaTask
  • ETLBox.DataFlow
    • AggregateColumn
    • ChangeAction
    • ColumnMap
    • CompareColumn
    • DataFlow
    • DataFlowBatchDestination<TInput>
    • DataFlowComponent
    • DataFlowDestination<TInput>
    • DataFlowExecutableSource<TOutput>
    • DataFlowJoinTarget<TInput>
    • DataFlowSource<TOutput>
    • DataFlowStreamDestination<TInput>
    • DataFlowStreamSource<TOutput>
    • DataFlowTransformation<TInput, TOutput>
    • DeleteColumn
    • DistinctColumn
    • ErrorSource
    • ETLBoxError
    • ExcelColumn
    • ExcelRange
    • GroupColumn
    • ICacheManager<TInput, TCache>
    • IDataFlowBatchDestination
    • IDataFlowBatchDestination<TInput>
    • IDataFlowComponent
    • IDataFlowDestination
    • IDataFlowDestination<TInput>
    • IDataFlowExecutableSource
    • IDataFlowExecutableSource<TOutput>
    • IDataFlowLogging
    • IDataFlowSource
    • IDataFlowSource<TOutput>
    • IDataFlowStreamDestination
    • IDataFlowStreamDestination<TInput>
    • IDataFlowStreamSource
    • IDataFlowStreamSource<TOutput>
    • IDataFlowTransformation<TInput, TOutput>
    • IDataFlowTransformation<TOutput>
    • IdColumn
    • IMergeableRow
    • JoinTarget<TInput>
    • MatchColumn
    • MemoryCache<TInput, TCache>
    • MergeableRow
    • MergeMode
    • MergeProperties
    • Network
    • ResourceType
    • RetrieveColumn
    • StreamMetaData
    • TypeInfo
    • UpdateColumn
  • ETLBox.DataFlow.Connectors
    • ConcurrentMemoryDestination
    • ConcurrentMemoryDestination<TInput>
    • CsvDestination
    • CsvDestination<TInput>
    • CsvSource
    • CsvSource<TOutput>
    • CustomBatchDestination
    • CustomBatchDestination<TInput>
    • CustomBatchSource
    • CustomBatchSource<TOutput>
    • CustomDestination
    • CustomDestination<TInput>
    • CustomSource
    • CustomSource<TOutput>
    • DbDestination
    • DbDestination<TInput>
    • DbMerge
    • DbMerge<TInput>
    • DbSource
    • DbSource<TOutput>
    • ExcelSource
    • ExcelSource<TOutput>
    • JsonDestination
    • JsonDestination<TInput>
    • JsonSource
    • JsonSource<TOutput>
    • MemoryDestination
    • MemoryDestination<TInput>
    • MemorySource
    • MemorySource<TOutput>
    • TextDestination
    • TextDestination<TInput>
    • TextSource
    • TextSource<TOutput>
    • VoidDestination
    • VoidDestination<TInput>
    • XmlDestination
    • XmlDestination<TInput>
    • XmlSource
    • XmlSource<TOutput>
  • ETLBox.DataFlow.Transformations
    • Aggregation
    • Aggregation<TInput, TOutput>
    • AggregationMethod
    • BatchTransformation
    • BatchTransformation<TInput, TOutput>
    • BatchTransformation<TInput>
    • BlockTransformation
    • BlockTransformation<TInput, TOutput>
    • BlockTransformation<TInput>
    • CachedBatchTransformation
    • CachedBatchTransformation<TInput, TOutput, TCache>
    • CachedBatchTransformation<TInput, TOutput>
    • CachedBatchTransformation<TInput>
    • CachedRowTransformation
    • CachedRowTransformation<TInput, TOutput, TCache>
    • CachedRowTransformation<TInput, TOutput>
    • CachedRowTransformation<TInput>
    • CacheMode
    • ColumnRename
    • ColumnRename<TInput>
    • CrossJoin
    • CrossJoin<TInput>
    • CrossJoin<TInput1, TInput2, TOutput>
    • CrossJoin<TInput1, TInput2, TOutput>.InMemoryDestination<TInput>
    • Distinct<TInput>
    • LookupTransformation
    • LookupTransformation<TInput, TSource>
    • LookupTransformation<TInput, TSource>.PartialDbCacheSettings
    • MergeJoin
    • MergeJoin<TInput>
    • MergeJoin<TInput1, TInput2, TOutput>
    • Multicast
    • Multicast<TInput>
    • RowDuplication
    • RowDuplication<TInput>
    • RowMultiplication
    • RowMultiplication<TInput, TOutput>
    • RowMultiplication<TInput>
    • RowTransformation
    • RowTransformation<TInput, TOutput>
    • RowTransformation<TInput>
    • Sort
    • Sort<TInput>
    • XmlSchemaValidation
    • XmlSchemaValidation<TInput>
  • ETLBox.Exceptions
    • ETLBoxException
    • ETLBoxFaultedBufferException
    • ETLBoxNotSupportedException
  • ETLBox.Helper
    • BatchLinq
    • BulkSqlGenerator<T>
    • DataTypeConverter
    • ExpandoJsonPathConverter
    • HashHelper
    • IDataTypeConverter
    • JsonPathConverter
    • JsonProperty2JsonPath
    • JsonProperty2JsonPath.MultipleResultsInPath
    • ObjectNameDescriptor
    • PropertyInfoExtension
    • SqlParser
    • StringExtension
  • ETLBox.Logging
    • AbortLoadProcessTask
    • CreateErrorTableTask
    • CreateLoadProcessTableTask
    • CreateLogTableTask
    • CustomTask
    • EndLoadProcessTask
    • ETLLogLayoutRenderer
    • GetLoadProcessAsJSONTask
    • GetLogAsJSONTask
    • LoadProcess
    • LogEntry
    • Logging
    • LogHierarchyEntry
    • LogTask
    • Package
    • ReadLoadProcessTableTask
    • ReadLogTableTask
    • ReadOptions
    • Sequence
    • Sequence<T>
    • StartLoadProcessTask

Class BatchTransformation<TInput>

A batch transformation will transform batches of data. The default batch size are 100000 rows. The batch transformation function allows you to process and modify each batch of data. You can use the BatchSize property to choose a smaller batch size. The batch size must always be smaller than the max buffer size. The default batch size are 1000 rows per batch. The batch transformation is a partial blocking transformation - it will always need at least enough memory to store a whole batch.

Inheritance
System.Object
LoggableTask
DataFlowComponent
DataFlowSource<TInput>
DataFlowTransformation<TInput, TInput>
BatchTransformation<TInput, TInput>
BatchTransformation<TInput>
BatchTransformation
Implements
ILoggableTask
IDataFlowLogging
IDataFlowTransformation<TInput, TInput>
IDataFlowSource<TInput>
IDataFlowSource
IDataFlowDestination<TInput>
IDataFlowDestination
IDataFlowComponent
Inherited Members
BatchTransformation<TInput, TInput>.TaskName
BatchTransformation<TInput, TInput>.SourceBlock
BatchTransformation<TInput, TInput>.TargetBlock
BatchTransformation<TInput, TInput>.BatchTransformationFunc
BatchTransformation<TInput, TInput>.BatchSize
BatchTransformation<TInput, TInput>.InitAction
BatchTransformation<TInput, TInput>.SuppressNullValueFilter
BatchTransformation<TInput, TInput>.CheckParameter()
BatchTransformation<TInput, TInput>.CheckAndAdjustBatchSize()
BatchTransformation<TInput, TInput>.InitComponent()
BatchTransformation<TInput, TInput>.CleanUpOnSuccess()
BatchTransformation<TInput, TInput>.CleanUpOnFaulted(Exception)
BatchTransformation<TInput, TInput>.WasInitActionInvoked
BatchTransformation<TInput, TInput>.InvokeInitActionOnce()
BatchTransformation<TInput, TInput>.InvokeBatchTransformationFunc(TInput[])
DataFlowTransformation<TInput, TInput>.TargetBlock
DataFlowSource<TInput>.SourceBlock
DataFlowSource<TInput>.LinkTo(IDataFlowDestination<TInput>)
DataFlowSource<TInput>.LinkTo(IDataFlowDestination<TInput>, Predicate<TInput>)
DataFlowSource<TInput>.LinkTo(IDataFlowDestination<TInput>, Predicate<TInput>, Predicate<TInput>)
DataFlowSource<TInput>.LinkTo<TConvert>(IDataFlowDestination<TInput>)
DataFlowSource<TInput>.LinkTo<TConvert>(IDataFlowDestination<TInput>, Predicate<TInput>)
DataFlowSource<TInput>.LinkTo<TConvert>(IDataFlowDestination<TInput>, Predicate<TInput>, Predicate<TInput>)
DataFlowSource<TInput>.LinkErrorTo(IDataFlowDestination<ETLBoxError>)
DataFlowComponent.MaxBufferSize
DataFlowComponent._maxBufferSize
DataFlowComponent.Predecessors
DataFlowComponent.Successors
DataFlowComponent.Completion
DataFlowComponent.WasComponentInitialized
DataFlowComponent.ReadyForProcessing
DataFlowComponent.WasLinked
DataFlowComponent.SetParent(DataFlowComponent)
DataFlowComponent.InternalLinkTo<T>(IDataFlowDestination, Object, Object)
DataFlowComponent.LinkBuffersRecursively()
DataFlowComponent.InitBufferObjects()
DataFlowComponent.CheckParameter()
DataFlowComponent.InitComponent()
DataFlowComponent.SetCompletionTask()
DataFlowComponent.RunErrorSourceInit()
DataFlowComponent.OnCompletion
DataFlowComponent.CompleteOrFaultBufferOnPredecessorCompletion(Task)
DataFlowComponent.CleanUpOnSuccess()
DataFlowComponent.CleanUpOnFaulted(Exception)
DataFlowComponent.Exception
DataFlowComponent.ErrorSource
DataFlowComponent.HandleCanceledOrFaultedBuffer()
DataFlowComponent.InternalLinkErrorTo(IDataFlowDestination<ETLBoxError>)
DataFlowComponent.ThrowOrRedirectError(Exception, String)
DataFlowComponent.ThrowError(Exception, String)
DataFlowComponent._loggingThresholdRows
DataFlowComponent.LoggingThresholdRows
DataFlowComponent.ProgressCount
DataFlowComponent.HasLoggingThresholdRows
DataFlowComponent.ThresholdCount
DataFlowComponent.WasLoggingStarted
DataFlowComponent.WasLoggingFinished
DataFlowComponent.NLogStartOnce()
DataFlowComponent.NLogFinishOnce()
DataFlowComponent.LogProgressBatch_AfterThrowOrRedirectError(Int32)
DataFlowComponent.LogProgress_AfterThrowOrRedirectError(Boolean)
LoggableTask.TaskType
LoggableTask.TaskName
LoggableTask._disableLogging
LoggableTask.DisableLogging
LoggableTask.TaskHash
LoggableTask.CopyLogTaskProperties(ILoggableTask)
System.Object.Equals(System.Object)
System.Object.Equals(System.Object, System.Object)
System.Object.GetHashCode()
System.Object.GetType()
System.Object.MemberwiseClone()
System.Object.ReferenceEquals(System.Object, System.Object)
System.Object.ToString()
Namespace: ETLBox.DataFlow.Transformations
Assembly: ETLBox.dll
Syntax
public class BatchTransformation<TInput> : BatchTransformation<TInput, TInput>, ILoggableTask, IDataFlowLogging, IDataFlowTransformation<TInput, TInput>, IDataFlowSource<TInput>, IDataFlowSource, IDataFlowDestination<TInput>, IDataFlowDestination, IDataFlowComponent
Type Parameters
Name Description
TInput

Constructors

BatchTransformation()

Declaration
public BatchTransformation()

BatchTransformation(Int32)

Declaration
public BatchTransformation(int batchSize)
Parameters
Type Name Description
System.Int32 batchSize

BatchTransformation(Int32, Func<TInput[], TInput[]>)

Declaration
public BatchTransformation(int batchSize, Func<TInput[], TInput[]> batchTransformationFunc)
Parameters
Type Name Description
System.Int32 batchSize
System.Func<TInput[], TInput[]> batchTransformationFunc

Implements

ILoggableTask
IDataFlowLogging
IDataFlowTransformation<TInput, TOutput>
IDataFlowSource<TOutput>
IDataFlowSource
IDataFlowDestination<TInput>
IDataFlowDestination
IDataFlowComponent
Back to top Crafted with <3 in Berlin
Contact & legal notice