Pervasive DataRush Use Case - E-Commerce Recommender System

Problem

  • Customer satisfaction and loyalty needs to be improved
  • Recommender system data is large and sparse
  • Training run time is too long
  • Current system doesn't meet accuracy targets

The Pervasive DataRush Solution

  • Apply parallelism to Data Mining using Pervasive DataRush
    • Uses commodity multicore hardware
    • Spawns as many threads as available
  • Use Pervasive dataflow model for large and sparse data
    • Data intensive without memory limitations
    • Horizontal partitioning based on number of cores
    • Sparse matrix reader with parallel parsing and extracting
  • Utilize state of the art Information Theoretic Coclustering Algorighm

Pervasive DataRush Benefits

  • Training runtime reduced from days to less than 17 minutes
  • Recommendation accuracy increased
  • System scalable across cores and data volumes
  • Rapid development environment