Text

The TextSource and TextDestination allow you to read or write data from/into a text file. The text connectors are part of the ETLBox core package.

Text source#

The text source let you read data from any text file. It will read every line from the source which can be transformed with a ParseLineAction - this allows you to parse the line into your data object as you like. As you need to define how a line in your file is converted into an object yourself, this source is not as convenient as other sources, but offers the most flexibility when reading text files in a non common format.

Let’s go through an example. If your text file look like this:

1--A
2--B
3--C

You could read this file into a dataflow with:

public class MyTextRow
{
    public int Id {get;set;}
    public string Text { get; set; }
}
        
TextSource<MyTextRow> source = new TextSource<MyTextRow>();
source.Uri = "inputFile.txt";
source.ParseLineAction = (line, progressCount) => {
    return new MyTextRow() {
        Id = int.Parse(line.Substring(0,1)),
        Text = line.Substring(3,1)
    };
};

Using dynamic objects#

Of course the TextSource also works with dynamic objects. The default implementation uses the ExpandoObject.

TextSource source = new TextSource();
source.Uri = "inputFile.txt";
source.ParseLineAction = (line, progressCount) =>
{
    dynamic row = new ExpandoObject();
    row.Id = int.Parse(line.Substring(0, 1));
    row.Text = line.Substring(3, 1);
    return row;
};

Using arrays#

Your input type could also be an array. If you define an array as input type, you can set the size of the array in the property ArraySize. Because the TextSource does the array initialization for you, this value will define the max number of elements accessible in the area. The default is 10.

TextSource<string[]> source = new TextSource<string[]>();
source.Uri = "inputFile.txt";
source.ParseLineAction = (line, progressCount) =>
{
    string[] row = new string[2];
    row[0] = line.Substring(0, 1);
    row[1] = line.Substring(3, 1);                
    return row;
};

Text destination#

The text destination let you create a text file from your incoming data. It allows you to define how the incoming data object is translated into a row in your text file destination. The text destination has a function that describe how the incoming row is converted into a string (similar to ToString()).

Assuming we have the same input data as above, the following code would convert this data back into a text file.

public class MyTextRow
{
    public int Id { get; set;}
    public string Text { get; set; }
}
        
TextDestination<MyTextRow> dest = new TextDestination<MyTextRow>("outputFile.txt");
dest.WriteLineFunc = (row,progressCount) => $"{row.Id}--{ro}.Text}";

Write header information#

If you want to add one or multiple header rows at the beginning of your destination file, you can use the WriteHeaderFunc function. This function must return a string that is written in the destination before the first row is processed. The header can span across multiple rows if needed.

TextDestination<MyTextRow> dest = new TextDestination<MyTextRow>("res/TextDestination/InitialWriteTest.txt");
dest.WriteHeaderFunc = () => "This is the first row"+Environment.NewLine+"Id  Value";
dest.WriteLineFunc = tr => $"{tr.Id}  {tr.Text}";

Using dynamic objects#

Instead of an object you can use the ExpandoObject with the default implementation.

TextDestination dest = new TextDestination("outputFile.txt");
    dest.WriteLineFunc =
    (row,progressCount) =>
    {
        dynamic r = row as ExpandoObject;
        return $"{r.Id}  {r.Text}";
    };

Using arrays#

This is the code for writing a string array input type into a file.

TextDestination<string[]> dest = new TextDestination<string[]>("outputFile.txt");
dest.WriteLineFunc = (row,progressCount) => $"{row[0]}--{row[1]}";