DataRush Technology

Pervasive DataRush is a Java implementation of dataflow. Here is more info from wikipedia.org:


Dataflow Programming: a simple, natural, powerful approach for programming multicore

"Dataflow languages contrast with the majority of programming languages, which use the imperative programming model. In imperative programming the program is modeled as a series of operations, the data being effectively invisible. This distinction may seem minor, but the paradigm shift is fairly dramatic, and allows dataflow languages to be spread out across multicore, multiprocessor systems for free."

"Dataflow languages promote the data to become the main concept behind any program. The data is now explicit. Operations consist of "black boxes" with inputs and outputs, all of which are always explicitly defined. They run as soon as all of their inputs become valid, as opposed to when the program encounters them."

"Whereas a traditional program essentially consists of a series of statements saying "do this, now do this", a dataflow program is more like a series of workers on an assembly line, who will do their assigned task as soon as the materials arrive. This is why dataflow languages are inherently parallel; the operations have no hidden state to keep track of, and the operations are all "ready" at the same time."

For years, chip manufacturers were content doubling CPU clock speed about every 18 months in order to keep up with the ever-increasing demand for compute power. Due to heat dissipation issues and "real-estate shortages" on the chip, most designers agreed this strategy could not be sustained - the multicore CPU industry was born.

The shift to multicore chip designs has been remarkably swift and decisive. Even more amazing, chip density has accelerated at mind-bending speeds from 2 to 4, 8 and even 46 cores on a single chip with 80-core chips planned for 2010. As a result, hardware platforms have completely outpaced the ability of current software designs to take advantage of their new-found compute power. How can today's software developers, unaccustomed to the complexities of concurrent programming, build applications that are multicore-aware?

Years in the making, Pervasive DataRush© technology is the only 100% Java framework that allows developers to quickly build highly parallel data processing applications for today's multicore hardware all without the need to deal with threading libraries, deadlock detection algorithms, or concurrent process design issues. DataRush comes with a rich library of out-of-the-box Java components that can be assembled into a series of data flow operations. Where custom components need to be added or extended, developers simply use the DataRush SDK to quickly build and extend their DataRush application.

Rich Component Library

  • Foundational library that includes I/O, sort, merge, join and other operators
  • Extensibility is provided by Java components that can be re-used and extended into higher-order components and/or assemblies

Parallel Processing Engine

  • The DataRush execution environment (DRE) allows for on-the-fly optimization of data processing applications
  • Any number of custom operators can be easily included by extending the execution classpath of DRE to include your JAR files
  • Debugging is made simpler with DRE execution statistics that provide component-level runtime statistics