A Customer Intelligence Platform: bringing customer insights in a CRM platform

Incorporating a metadata layer, and a Data Lake Datasets composed of discrete objects, like image collections, scanned documents or free text is a poor match for the traditional database environment. These requirements compel rethinking of basic assumptions about data architecture and system design, assumptions that have been present for two decades. It is no longer sufficient to use only a relational database and an ETL tool, nor is there a single unified data model for all data. Instead there are many discrete data sets that can be integrated as needed, stored in their original form or in various stages of integration all the way through to the heavily standardized and quality-assured data one finds in a data warehouse. This paper present Apache Spark as a computation Engine designed to solve the challenges related to data gravity, the fact that Services and applications that use data tend to bring even more masses of data.