DataSync sequence number¶
Data Sync determines which content and in which order to import it into the
database using a sequence number. This number is auto-generated for each
Application, Source and Destination combination in an incremental way, for
both import and export operation individually.
Execution implications¶
As stated before, the sequence number is used to determine what content to
import (as explained below) and in which order it must be done.
The ImportEngine checks the sequence number of each content to insert, making
sure its sequence number equals the last sequence increased by one.
If multiple contents were to be imported at once, the ImportEngine will only
import the contents that comply with this sequencing control, and return errors
for all those that doesn't.
Info
Example: If the last import sequence number for certain application, source and destination is 100, and the contents to import have the sequence numbers 101, 102, 104 and 105; the engine will try to only import the contents up to 102 and return error for 104 and 105 (since the content number 103 is missing).
Each import process result is saved in the database of DataSync, having the sequence number of the content associated. The last sequence number for the import is obtained from these results, taking the biggest number from the valid (a.ka. whit no major errors) records.
Info
For the previous example, in the next import operation the last sequence number will be 102, and the contents to import the ones with a sequence number bigger than that; if the contents are 104 and 105 they will return errors until either the content 103 appears or the sequence control is set to be ignored (explained below).
Ignore sequence control¶
The sequence control can be ignored in an import operation by setting the
IgnoreSequenceNumber option in the parameter as true when using the
ReplicationAppService, this way the contents are not validated to follow a
sequence, but they will be imported following the order.
Info
For the previous example, the engine will import up to content 105, following their order.
Gateway implications¶
The sequence number of the contents exported are generated in the
ExportEngine and used in the Gateway services in their different ways.
File system¶
The FileSystemGatewayService uses the sequence number to generate the names
of the files and also stores it within their content.
When this service is asked to obtain the contents to import from the files, it filters them using the sequence number in their names, taking all those files whose number is bigger than certain value, which is calculated for each combination of application, source and destination.