Understanding Named Pipe Access Module data transfer
The Teradata Named Pipe Access Module (NPAM) provides an inter-process communication link between a writer process (such as FastExport) and a reader process (such as FastLoad).
NPAM – Data transfer
The below block diagram (which is self-explanatory) depicts the transfer of data between a writer process (such as the Teradata FastExport utility) and a reader process (such as the Teradata FastLoad utility). The reader process interacts with Named Pipes Access Module via the Teradata Data Connector Module. It is not necessary for the writer process to use the Teradata Data Connector to interact with the NPAM.
The data transfer between writer process and reader process is performed as follows:
- The reader process loads and initializes the NPAM library via the Teradata Data Connector. This step includes the creation of the named pipe.
- The writer process writes the data (in the same format that the reader process expects) on to the named pipe. Note that the writer process does not require an instance of the Data Connector Module. The writer process can be any third-party application that supplies data through the named pipe in the format the reader process expects.
- The NPAM tracks the data flow, copies the inbound data into a fallback data file. The data is read by the reader process. Note that the NPAM creates and manage the fallback data file throughout the data transfer activity. Note that the fallback data file is deleted by the NPAM after the completion of the job.
- The data transfer is performed by the writer and reader processes until the end-of-file (EOF) is reached.
- If the reader process determines that it must fall back to an earlier point in the data stream, it issues the standard File Set Position access command to NPAM (via the Teradata Data Connector), which then supplies subsequent reads from the data it saved in the fallback data file.
- The reader and the writer processes should use the same name of the Named pipe for the read operation and write operation respectively.
- The reader and the writer processes should use the same format of the data being transferred via the Named pipe. That is, the format of the data written by the writer process should be understandable by the reader process.
Example to illustrate the usage of the NPAM
The reader process (such as the Teradata TPUMP) can use the following statement in the job script to use the NPAM module for the data transfer.
.IMPORT INFILE np1 /* where “np1” is the name of the Named pipe used */ LAYOUT LAY1A /* where “LAY1A” is the name of the data LAYOUT used */ APPLY LABELA /* where “LABELA” is the name of the LABEL used */ FORMAT fastload /* where “fastload” is the type of data FORMAT used */ axsmod np_axsmod.so 'll=5 ld=. b=32000 fd=. cfd=n sc=2' ; /* “np_axsmod.so” is the NPAM library name */ /* ‘Quoted string’ given adjacent to the NPAM library is the Initialization string which is used to Initialize the NPAM module */
Writing data via the Data Connector
The writer process (such as Teradata FastExport) can use the following statement in the job script to write the data (to be transferred) on to the Named pipe. The Teradata FastExport is dependent on the Data Connector to perform the I/O operations. Hence the data to be transferred is written by the Teradata FastExport via the Data Connector module on to the Named pipe.
.EXPORT OUTFILE np1; /* where “np1” is the name of the Named pipe. */
Writing data without using the Data Connector
The data can be written on to the Named pipe by using simple “cat” command as mentioned below.
$ cat data_file > np1
- “data_file” contains the data to be transferred to the reader process.
- “np1” is the name of the Named pipe being used for the data transfer.