Extreme streaming: business optimization driving algorithmic challenges

ABSTRACT

Organizations are striving for competitive advantage. As a result, business optimization is being pushed to new heights in terms of volume and speed. Areas such as customer profitability, campaign profitability or customer insight for better service are driving new analytical challenges as well as new algorithms over large volumes of data.

Using Telecom as an example, this talk will discuss business demands which are driving an evolution of analytics over extremely high volumes of streaming data being ingested into a warehouse -- this business direction is forcing algorithms to evolve. In this environment, we are forced to deal with (1) massive continuous ingest rates, data changes/updates while the same data is being consumed by applications, users and processes, (2) dealing with scalability without disruption, and (3) overcoming the physical limits of the infrastructure (e.g., IO).

This talk will address both how these technical challenges are being addressed at a high-level, as well as known problem areas for further research. This talk will also highlight issues of new algorithmic approaches arising from the business needs and the research issues associated with them. Examples include (1) streaming analytics, (2) social analytical algorithms integrated into data mining approaches within these continuously streamed environments and (3) new data types such as voice and textual analytics for customer calling pattern and churn analysis.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.8 Database applications

Additional Classification:
  H. Information Systems
  H.4 INFORMATION SYSTEMS APPLICATIONS
      H.4.2 Types of Systems

General Terms:
Algorithms, Design, Management, Performance

Keywords:
business intelligence