Gateway

In DataSync, the Gateway refers to the layer in charge of providing repositories to store the data obtained from an export operation in certain Site and retrieve it back to be used in an import operation. This layer is also in charge of performing any needed work over these repositories after/before the replication processes.

Providers

The Gateway layer is an abstraction that allow us to define different providers with their own storage and transport logic:

File-System

This gateway provider is in charge of transporting the data through the use of file-system, this means that the repositories generated by this provider stores the data in Mapped-files. During the different replication process it is also in charge of manipulate these files in the following way:

To dispatch the information, it is initially stored in files. They are compressed into a package which is then split into smaller segments before being finally sent.
To retrieve the information, the provider first combines the segments back into the original package file and decompress them, finally reading their content.

Mapped-file

A MappedFile refers to a file that contains an index inside it, which is used to identify certain "areas" of the file through a unique key. This index is then used during reading operations to access the data stored inside certain area directly.
This allows us to obtain the data to be used (i.e: in an Import operation) without the need of loading the entire file content into memory, improving the performance of the engine.