Improvised methods for tackling big data stream mining challenges: case study of human activity recognition

Simon Fong, Kexing Liu, Kyungeun Cho, Raymond Wong, Sabah Mohammed, Jinan Fiaidhi

Research output: Contribution to journalArticlepeer-review

9 Scopus citations

Abstract

Big data stream is a new hype but a practical computational challenge founded on data streams that are prevalent in applications nowadays. It is quite well known that data streams that are originated and collected from monitoring sensors accumulate continuously to a very huge amount making traditional batch-based model induction algorithms infeasible for real-time data mining or just-in-time data analytics. In this position paper, following a new data stream mining methodology, namely stream-based holistic analytics and reasoning in parallel (SHARP), a list of data analytic challenges as well as improvised methods are looked into. In particular, two types of decision tree algorithms, batch-mode and incremental-mode, are put under test at sensor data that represents a typical big data stream. We investigate whether and to what extent of two improvised methods—outlier removal and balancing imbalanced class distributions—affect the prediction performance in big data stream mining. SHARP is founded on incremental learning which does not require all the training to be loaded into the memory. This important fundamental concept needs to be supported not only by the decision tree algorithms, but by the other improvised methods usually at the preprocessing stage as well. This paper sheds some light into this area which is often overlooked by data analysts when it comes to big data stream mining.

Original languageEnglish
Pages (from-to)3927-3959
Number of pages33
JournalJournal of Supercomputing
Volume72
Issue number10
DOIs
StatePublished - 1 Oct 2016

Keywords

  • Big data
  • Data stream mining
  • Resampling
  • Sensor data
  • Very fast decision tree

Fingerprint

Dive into the research topics of 'Improvised methods for tackling big data stream mining challenges: case study of human activity recognition'. Together they form a unique fingerprint.

Cite this