Today's scientific processes heavily depend on fast and accurate analysis of experimental data. Scientists are routinely overwhelmed by the effort needed to manage the volumes of data produced either by observing phenomena or by sophisticated simulations. As database systems have proven inefficient, inadequate, or insufficient to meet the needs of scientific applications, the scientific community typically uses special-purpose legacy software. With the exponential growth of dataset size and complexity, application-specific systems, however, no longer scale to efficiently analyze the relevant parts of their data, thereby slowing down the cycle of analyzing, understanding, and preparing new experiments.
In this talk I will illustrate the problem with a challenging application featuring brain simulation data and will show how the problems from neuroscience translate into interesting data management challenges. Finally I will also use the example of neuroscience to show how novel data management and, in particular, spatial indexing and navigation have enabled today's neuroscientists to simulate a meaningful percentage of the human brain.