My boss and I attended the Intel Developer Forum last week in SF. There were 6100 attendees, and an amazing array of special Intel hardware initiatives discussed and exhibited, from extending battery life, to embedded systems, to visual computing to 3D movies. The explosion of new hardware development was awesome.

There are always a near infinite number of challenges with launching a new product; even more so if the product is conceptually different than the others in the space. Following the Theory of Constraints, you have to focus on the current constraining issue until it is resolved and you can move on to the next one.

The first release candidate of DataRush is now available to download. The main new feature of this release is the support of dataflow graph composition in Java! In the previous releases of DataRush, dataflow graph composition was accomplished through a combination of an XML scripting language and composition helpers known as customizers. This led to a quagmire of XML and Java code that was not always easy to follow. But no more!

Please feel free to download the release and experiment. There are several sample applications included in the release that provide a quick overview of how to use DataRush to build highly scalable, data-intensive applications. The Javadoc is extensive and provides good reference documentation to help you take DataRush to the next step.

We'll be updating the site with bug fix releases and new documentation as they become available. So please, check back or better yet, use RSS to keep track of announcements.

Enjoy!

The following is a new white paper written by our Chief Technologist, Jim Falgout. I have copied the text here, but the clean .pdf is also attached and is more complete:

Pervasive DataRush™: An Architecture Paper Describing the Highly Parallel Dataflow Framework

A Pervasive Software White Paper
July 2008

DataRush is sufficiently sophisticated (or at least different) that understanding it takes several passes. I am writing a series of posts aimed toward exploring what DataRush is and is not. This should give the passing programmer a better feel for what DataRush might do for them and how it fits into the broader scheme of concurrent programming techniques.

So, back to the question at hand; is DataRush dataflow? The answer is obvious: that depends upon what "dataflow" is.

According to [CTMCP], dataflow behavior results when expressions contain unbound variables. When execution reaches such an expression, the program simply pauses, awaiting a value. If at some point in the future another thread binds a value to the variable, the program picks up where it left off. They call variables with these characteristics dataflow variables. A simple example in Oz (from page 60) follows:

local X Y Z in
  X=10
  if X>=Y then Z=X else Z=Y end
end

Note that X is declared, then immediately bound to a value. The purpose of the expression is to bind Z. However, that leaves Y unbound. From the expression above, we simply cannot tell what value Y should take and, unlike some other programming languages, Oz does not arbitrarily specify a default value.