ETLBox

A lightweight ETL (extract, transform, load) toolbox and data integration library for .NET

Try for free. Get commercial licenses at low rates.

Highly customizable

Tired of using User Interfaces where complex requirements are almost impossible to implement? With ETLBox it's easy to write code that fits your unique needs. Plug in your own logic or modify existing behavior to your specific requirements.

Test driven data flows

Always wanted to create your ETL jobs test driven? Still wondering if your data flow will still work after you latest change? Look no further. ETLBox integrates with all common test framework like xUnit, NUnit or MSTest. Now you can write test for any part or logic of your data pipeline.

Develop locally

Use the toolset that you and your team is already familiar with. Develop your data pipeline locally with your favorite development and debugging tools. No need to learn a new programming language.

Manage your changes

Track your changes with Git (or other source code controls), integrate with your current code review process and connect with your existing CI/CD platform.

Build ETL in .NET

Code your ETL with your favorite .NET language fitting your team’s skills and that is coming with a mature toolset.

Runs everywhere

ETLBox runs on Linux, macOS, and Windows. It is written in the current .NET Standard and successfully tested with the latest versions of .NET.

Fast and efficient

Process your data in-Memory:
Code your ETL with your favorite .NET language fitting your team’s skills and that is coming with a mature toolset.

Made for big data

Process any workload:
ETLBox relies on Microsoft's System.Threading.TPL.Dataflow library and is designed to process efficiently big amounts of any data.

Connect everything

Process data from anywhere:
ETLBox supports a wide range of Sql & NoSql databases, flat files, web services, and more - perfect as a foundation for your tailor made data integregation platform.

ETL made easy

Use a simple syntax to create easy or complex data flows. Works with strongly typed and dynamic objects.

string mySqlConnString =
  @"Server=localhost;Database=ETLBox;Uid=user;Pwd=password;";
string postgresConnString =
  @"Server=.;Initial Catalog=ETLBox;Trusted_Connection=true;";

var sourceCon = new MySqlConnectionManager(mySqlConnString);
var destCon = new PostgresConnectionManager(postgresConnString);

var source = new DbSource(sourceCon, "SourceTableName");
var dest = new DbDestination(destCon, "DestTableName");
//Use dest.ColumnMapping to change column name mapping

source.LinkTo(dest); //Link the nodes in the network
await Network.ExecuteAsync(source); //Start the flow

string connString =
  @"Source=.;Trusted_Connection=true;Initial Catalog=ETLBox;"
var destCon = new SqlConnectionManager(connString);

var source = new CsvSource(sourceCon, "SourceData.csv");
var dest = new DbDestination(destCon, "DestinationTable");

//Default mapping: Csv header and column name match (case-sensitive)
//Override mapping like this:
dest.ColumnMapping = new[] {
    new DbColumnMap() {
      DbColumnName = "Id", PropertyName = "OrderNumber"
    }
};

source.LinkTo(dest); //Link the nodes in the network
await Network.Execute(source); //Start the flow

public class MyMergeRow : MergeableRow
{
    [IdColumn]
    public long Key { get; set; }
    [CompareColumn]
    [UpdateColumn]
    public string Value { get; set; }
    [DeleteColumn(true)]
    public bool DeleteThisRow { get; set; }
}

DbSource<MyMergeRow> source = new (connection, "SourceTable");
DbMerge<MyMergeRow> mergeDest = new (connection, "DestinationTable");
merge.MergeMode = MergeMode.Full;
merge.CacheMode = CacheMode.Partial;
source.LinkTo(merge);
Network.Execute(source);

Watch our short explainer video

This video is also available on youtube, if you want to like or share it.

Ask ChatGPT about ETLBox

Hi ChatGPT, could you give me an introduction into ETLBox?

Yes, of course!

ETLBox is a commercial, .NET-based library for Extract-Transform-Load (ETL) processes. ETL is a critical aspect of data management, and is used to extract data from a variety of sources, clean and transform it into a desired format, and then load it into a target system, such as a data warehouse or a database. ETLBox provides a set of simple, yet powerful components that can be used to automate these tasks.

The key benefits of using ETLBox include:

Simplicity: ETLBox provides an easy-to-use API that makes it simple to implement ETL processes.
Flexibility: ETLBox supports a wide range of data sources and targets, and provides the ability to transform data in many ways, such as by performing lookups, splitting columns, and aggregating data.
Performance: ETLBox is designed for high performance and can handle large amounts of data efficiently.
.NET support: ETLBox is written in C# and is fully integrated with the .NET framework, making it a natural choice for developers who are already familiar with .NET.

In conclusion, ETLBox is a great choice for anyone who needs to implement ETL processes and wants a powerful, yet simple, solution that is commercially supported.

What is ETL

ETL is the foundation of modern Business Intelligence application. It separates the analysis part from everything that comes before it. ETL stands for extract, transform and load, and describes a three-step process:

Extracting data is about acquiring it from a particular source and doing so in a way that all relevant data is collected. For ETL to work well, data needs to come directly from the source and in its rawest form.

Transforming data requires it to be stripped of its various formats, cleaned and harmonized. Put differently, it needs to become easy for the end user to read. Ensuring the highest possible data quality is crucial during this process, including detecting and removing duplicates and sorting the data into relevant, predetermined categories.

In the end, your ETL tool will help harmonize your data and load it into a target location, which essentially is a business intelligence (BI) or database tool of choice. There you can find it ready for your analysis.

ETLBox core concepts

ETLBox is a powerful and versatile C# library for performing ETL operations. With its simple and intuitive API, you can quickly and easily extract data from various sources, transform it into the desired format, and load it into your target system. Whether you need to move data from one database to another or process large amounts of data for business intelligence purposes, ETLBox provides a comprehensive set of tools to streamline your work. It's highly customizable and flexible, so you can easily adapt it to your specific needs.

What you can do with ETLBox

Here are a few things you can do with ETLBox:

Convert incoming data to a common format.
Prepare data for analysis and visualization.
Migrate between databases.
Share data processing logic across web apps, batch jobs, and APIs.
Power your data ingestion and integration tools.
Consume large CSV, Excel or Json files.

Loved by our customers across the globe

Data integration

Data integration solves the problem of moving, transforming, and consolidating information from various sources like databases, applications, files, and web services. Bringing them together enables you to cleanse, standardize, de-duplicate, manipulate, and synchronize them. Data integration can play an important role in any project, and it can be critical to have a rock-solid solution in place that is able to integrate numerous applications, services and databases. ETLBox as an Extract, transform, and load (ETL) tool is a perfect fit when you need to create a data integration solution.

Common data API

ETLBox provides you with a single API for working with data. The API treats all data the same regardless of their source. By developing your applications against a single API, you can use the same components to process data regardless of whether they're coming from a database, Excel file, or 3rd-party API.

Embed inside your App

ETLBox fits well within your applications and services. It is a very lightweight library and has a very small footprint. It's also complication free - requiring no servers, installation, or config files. You just add it as a package reference into your app and start using it.

In-flight Processing

ETLBox can run completely in-memory. In most cases, there's no need to store intermediate results in temporary databases or files on disk. Processing data in-memory, while it moves through the pipeline, can be more than 100 times faster than storing it to disk to query or process later.

Flexible componentens

ETLBox comes with a set of components and tasks that simplify to read data from various sources, transform it on the fly and write it into different destinations. But all components offer you the ability to add your own logic in C# - and you can benefit here from the full power of the .NET framework. Even if your own data source or destination is not supported out of the box, you can easily add it yourself by extending the CustomSource or CustomDestination components.

100% .NET

ETLBox is built on .NET and runs with the latest version of .NET, .NET Core and .NET Framework. You can write data flows and transformations in C# or any of the other .NET languages you know (e.g. VisualBasic or F#). Also use your existing tools, IDEs, containers, and libraries. Being built on .NET means ETLBox can run on all servers, operating systems, and environments.

Explore Some of Our Supported Connectors:

These are just a few of the many connectors ETLBox supports. Whether you're integrating data from popular databases, cloud services, or other platforms, ETLBox makes connecting and transforming your data simple and efficient. Explore our full range of connectors to discover even more possibilities for your data workflows.

...or create your own with our CustomConnector!

Build Your Dataflows with Code

Take the next step in simplifying your data integration. With ETLBox, you can easily write code to set up, connect, and transform your data exactly the way you need.
Click below to begin crafting your ETL processes with our powerful C# library.

Ready to start?