ETLBox ETLBox
API
Try now!

CosmosDestination<TInput>

Class CosmosDestination<TInput>

A data flow destination for inserting data into a couchbase bucket.

Inheritance
object
LoggableTask
DataFlowComponent
DataFlowDestination<TInput>
DataFlowBatchDestination<TInput>
CosmosDestination<TInput>
CosmosDestination
Implements
Inherited Members
Namespace: ETLBox.Azure.CosmosDb
Assembly: ETLBox.Azure.CosmosDb.dll
Syntax
    public class CosmosDestination<TInput> : DataFlowBatchDestination<TInput>, IDataFlowBatchDestination<TInput>, IDataFlowDestination<TInput>, IDataFlowBatchDestination, IDataFlowDestination, IDataFlowComponent, ILoggableTask
Type Parameters
NameDescription
TInput

Type of ingoing data.

Constructors

CosmosDestination()

Declaration
    public CosmosDestination()

CosmosDestination(int)

Declaration
    public CosmosDestination(int batchSize)
Parameters
TypeNameDescription
intbatchSize

CosmosDestination(string, string, string)

Declaration
    public CosmosDestination(string connectionString, string databaseName, string containerName)
Parameters
TypeNameDescription
stringconnectionString
stringdatabaseName
stringcontainerName

Properties

AccountEndpoint

Optional: An Azure Cosmos Db account endpoint used when opening the connection (works only in combination with AuthKeyOrResourceToken) ConnectionString, AzureKeyCredential and AzureTokenCredential have no effect then.

Declaration
    public string AccountEndpoint { get; set; }
Property Value
TypeDescription
string

AuthKeyOrResourceToken

Optional: An Azure Authorization key or resource token which is used when opening the connection (works only in combination with AccountEndpoint) /// ConnectionString, AzureKeyCredential and AzureTokenCredential have no effect then.

Declaration
    public string AuthKeyOrResourceToken { get; set; }
Property Value
TypeDescription
string

AzureKeyCredential

Optional: An Azure Key Credential used when opening the connection (works only in combination with ConnectionString)

Declaration
    public AzureKeyCredential AzureKeyCredential { get; set; }
Property Value
TypeDescription
AzureKeyCredential

AzureTokenCredential

Optional: An Azure Token Credential used when opening the connection (works only in combination with ConnectionString)

Declaration
    public TokenCredential AzureTokenCredential { get; set; }
Property Value
TypeDescription
TokenCredential

ConnectionString

The connection string which points to the Azure Cosmos Db. The connection string may contain all relevant connection information. Please also specify the DatabaseName and ContainerName. Optional: You can also provide either an AzureKeyCredential or AzureTokenCredential along with the connection string. Alternatively, you can use the AccountEndpoint and AuthKeyOrResourceToken to establish a connection.

Declaration
    public string ConnectionString { get; set; }
Property Value
TypeDescription
string

ContainerName

The container name of the Azure Cosmos Db destination

Declaration
    public string ContainerName { get; set; }
Property Value
TypeDescription
string

CosmosClient

Optional: Instead of providing the ConnectionString and/or other credentials, you can also create the CosmosClient manually in your code and assign it to this property. This client will then be used, and no new client won't be created. Please make sure bulk insert is allowed in the client.

Declaration
    public CosmosClient CosmosClient { get; set; }
Property Value
TypeDescription
CosmosClient

DatabaseName

The database name of the Azure Cosmos Db destination

Declaration
    public string DatabaseName { get; set; }
Property Value
TypeDescription
string

SetItemActionOptions

By default, all data is created/inserted into the destination. For each record, you can influcene the item action (e.g. to change it to upsert, replace or delete). Replace and Delete will need to set a partition key for each record via the SetItemActionOptions. For Delete you will also need to set an id to identify the record to be deleted.

Declaration
    public Action<ItemActionOption, TInput> SetItemActionOptions { get; set; }
Property Value
TypeDescription
Action<ItemActionOption, TInput>
Examples
dest.SetItemActionOptions = (options, item) => {
    options.ItemAction = ItemAction.Delete;  //or ItemAction.Replace or ItemAction.Upsert
    options.Id = item.Id; //only for Deletes                
    options.PartitionKey = new PartitionKey(item.PartitionKey);
};

UseTransactionalBatch

By default, data is inserted as bulk. If you prefer to insert data as part of a transactional batch, set this property to true. The maximum allowed batch size is then 100 (due to the limitation of Cosmos Db). This works best if all data has the same partition key. If the incoming data has different partition keys, the data is inserted in multiple batches, grouped by each partition key. Make sure to add a partition key for each record using the SetItemActionOptions action. If any of the records in the batch is flawed, the whole batch will not be inserted. You can redirect erroneous batches using 'LinkErrorsTo()' when linking your components.

Declaration
    public bool UseTransactionalBatch { get; set; }
Property Value
TypeDescription
bool
Examples
dest.UseTransactionalBatch = true;
 dest.SetItemActionOptions = (config, item) => config.PartitionKey = new PartitionKey(item.Block);

Methods

BulkInsertData(TInput[])

Declaration
    protected override void BulkInsertData(TInput[] data)
Parameters
TypeNameDescription
TInput[]data
Overrides

CheckParameter()

Declaration
    protected override void CheckParameter()
Overrides

FinishWrite()

Declaration
    protected override void FinishWrite()
Overrides

PrepareParameterForCheck()

Declaration
    protected override void PrepareParameterForCheck()
Overrides

PrepareWrite()

Declaration
    protected override void PrepareWrite()
Overrides

Implements

Edit this page on GitHub
Prev
AttributeType
Next
CosmosDestination