There are Big Data problems today. Usually when researchers talk about Big Data, however, the data isn't that big. An inefficient solution stack based on direct products over data-sets (Map-Reduce/Hadoop), cramming data (the square peg) into whatever data-store is available (the round hole), and moving data around between data-severs and
computer-servers: all of this is OK if Big Data is not that big. But what happens when the data in Big Data actually gets big?
In this talk we will discuss how Big Data software solution stacks must evolve to address future Big Data problems. Our guiding principles are that in a world where Big Data is really big:
* One size does not fit all; data must match the data store
* Data movement is everything; move queries and processing to the data
* Visualization must be a first class citizen of the Big Data solution stack
We are developing a reference implementation of the sort of Big Data solution stack we envision will play a key role in the future. We call this the BigDAWG solution stack. The research behind BigDAWG is occurring within the Intel Science and Technology Research Center at MIT with collaborators from Brown University, University of Washington, University of Tennessee, and Portland State University.
