NCPL Data Pipeline is a web service that helps you reliably process and move data between different NCPL compute and storage services, as well as on-premise data sources, at specified intervals. With NCPL Data Pipeline, you can regularly access your data where it’s stored, transform and process it at scale, and efficiently transfer the results to services such as Amazon S3, Amazon RDS, Amazon DynamoDB, and Amazon EMR.
Data Pipeline helps you easily create complex data processing workloads that are fault tolerant, repeatable, and highly available. You don’t have to worry about ensuring resource availability, managing inter-task dependencies, retrying transient failures or timeouts in individual tasks, or creating a failure notification system. Data Pipeline also allows you to move and process data that was previously locked up in on-premise data silos.