Pervasive DataRush - Unleash the Power of Multicore

Dataflow implementation in Java

mwalker — Wed, 23 Jul 2008 10:24:42 -0500

DataRush is sufficiently sophisticated (or at least different) that understanding it takes several passes. I am writing a series of posts aimed toward exploring what DataRush is and is not. This should give the passing programmer a better feel for what DataRush might do for them and how it fits into the broader scheme of concurrent programming techniques.

So, back to the question at hand; is DataRush dataflow? The answer is obvious: that depends upon what "dataflow" is.

According to [CTMCP], dataflow behavior results when expressions contain unbound variables. When execution reaches such an expression, the program simply pauses, awaiting a value. If at some point in the future another thread binds a value to the variable, the program picks up where it left off. They call variables with these characteristics dataflow variables. A simple example in Oz (from page 60) follows:

local X Y Z in
  X=10
  if X>=Y then Z=X else Z=Y end
end

Note that X is declared, then immediately bound to a value. The purpose of the expression is to bind Z. However, that leaves Y unbound. From the expression above, we simply cannot tell what value Y should take and, unlike some other programming languages, Oz does not arbitrarily specify a default value.

So what good are dataflow variables? Well, they facilitate the concurrent technique of dataflow programming. Combined with a single-assignment store (variables may be bound at most one time), they lead to the nice property that it doesn't matter in what order we evaluate simultaneously executing expressions. You can view dataflow variables as one-to-many channels allowing the thread in which the variable is bound to send a message ("wake up, the variable's value is ready!") to any waiting threads.

Now you can certainly dig a little deeper into the various meanings of dataflow. You'll find the dataflow variable of [CTMCP] is a concurrent logic variable, which is a promise updated by unification. The name derives from the variable representing the promise of a value to come, which may be fulfilled by any thread within the program. Good old Java provides an interface and implementations for a related idea, a future, in java.util.concurrent.

You'll also find dataflow architecture refers to a non-von Neumann way of coordinating the processing of instructions in a computer. Dataflow architectures don't iterate across instructions with an instruction pointer but fire off units of computation only when their inputs become ready. Mapping these units of computation and their inputs to the threads and dataflow variables discussed above, you can see the relationship between the two: you aren't controlling the order of execution of computations but merely interrelating them by the data they reference.

And this (finally!) brings us to a discussion of what ways DataRush is an implementation of dataflow in Java. Dataflow variables are a rather implicit and clean way of constructing what amounts to channels passing data amongst disparate threads in your program. The dataflow of DataRush is far more explicit: you implement and connect nodes (like the threads of Oz or computation units of a dataflow architecture) by writing Java classes. The following snippet of code mimics the behavior of the Oz expression above:

public class BindProcess extends DataflowNodeBase {
  private IntInput y;
  private IntOutput z;
  private int x;

  public BindProcess(IntFlow source, int x) {
    this.x = x;
    y = newIntInput(source, "y");
    z = newIntOutput("z");
  }

  public void execute() {
    while (y.stepNext()) {
      if (x >= y.asInt()) {
        z.push(x);
      } else {
        z.push(y.asInt());
      }
    }
    z.pushEndOfData();
  }
}

Like the dataflow of Oz, all downstream nodes listening to the same output channel block upon asking for data. Unlike Oz, the input cannot simply be used in an expression but must be explicitly stepped in each node, then polled for a value. Here, the stepNext() method causes the thread running an instance of BindProcess to block if y contains no data. Once data accumulates, the executor wakes the thread and asInt() retrieves the value at the head of the queue. The results of the node's computation must then be explicitly pushed to the z output.

Unlike dataflow variables, DataRush ports transfer batches of data thought of as a continuous stream flowing through the network; DataRush trades what amounts to multiple assignment to the port for requiring the programmer to explicitly manage the iteration through the incoming values.

Bear in mind as long as the instance of BindProcess steps y to end of data (or detaches from it), pushes end of data on z, and meets the cardinality requirements of downstream nodes receiving z, the execute() method may do whatever the programmer desires. Thus the dataflow of DataRush is a sort of dataflow by convention: the while loop, or some mechanism for stepping through a process' inputs like it, is central to the flow of data through the network.

DataRush is not a dataflow programming language, but a framework for constructing software according to a dataflow architecture. In this respect, it is far more similar to Morrison's flow-based programming than to the dataflow of Oz.

[CTMCP] Concepts, Techniques, and Models of Computer Programming. Van Roy and Haridi. 2004.

Diminishing returns from virtualization will affect larger core count server sales

steveh — Mon, 23 Jun 2008 11:33:05 -0500

I just got back from an HP show where I had some interesting conversations regarding the crisis that hasn't seemed to have happened yet.

The key to the Sherlock Holmes mystery, "The Hound of the Baskervilles" was this: Why the dog didn't bark?

Why hasn't the so-called multicore crisis been seen as a crisis? Why isn't there any angst over lack of parallelized applications and engines? Why is IT still buying multicore servers when they aren't doing development that can utilize that architecture?

One answer has to do with virtualization. The hypothesis is that the initial transition to multicore has been welcomed with open arms as IT managers sought a way to reduce power and cooling requirements, and multicore servers combined with virtualization software made that easy, inexpensive, and readily available. "Yes boss, we had a problem with heat and power, but we have taken care of it." Multicore crisis averted, because these boxes helped solve a real problem for IT. A different problem, but still a real one.

So the sales of 4 and 8 core servers have been robust and the hardware guys are happy, the software guys are happy, and IT is happy - what could possibly go wrong?

It is that these sales will not continue at larger core counts. It doesn't take an MBA to understand that there are diminishing returns -- there is no reason to go to a 32 core server when 4 8-core servers will do the job. Those 4 are less expensive than the single 32-core machine, we have put our eggs in 4 baskets instead of one, and we already are following that pattern. So the crisis that didn't happen when single processor servers became multicore has been delayed until now, when there is no obvious reason for IT to buy the larger core count boxes.

For companies like HP, AMD, and Intel, this bump in the road is coming later than predicted, but it is coming -- virtualization did not eliminate it. The smart people in these companies are beginning to ask if the forestalled crisis is arriving, and how they can help IT continue to gain business value from continued investment in larger core count machines.

What are you going to do with a 32-core machine? Break it up into 32 app servers? Or take advantage of the power of concurrent programming to enable high performance commercial computing (HPCC)?

the zettaflop, the yottaflop and the xeraflop

steveh — Mon, 9 Jun 2008 11:59:05 -0500

At a recent industry conference, it was shown that the biggest growth area in computing is HPC, or High Performance Computing. This was surprising, even to those in this field, as it has historically been a fairly small, insular, academic area populated by geeky professors and hard-working grad students, all using Fortran.

But a new phrase has become common, HPCC, or High Performance Commercial Computing. This is an attempt to make a distinction between the academic world and the business world, while still acknowledging their joint interests. But how closely aligned are these two spaces?

From our perspective, not so much. Academics use clusters, running Fortran. A speaker at a Supercomputing conference I attended pounded the lectern and declared, "Fortran is good enough, dammit!" Academics use grad students, who are very smart, are essentially immobile, and are virtually slave labor, and while performance matters to all, academics don't really care how long the overall project takes, as you get 7 years to write your dissertation.

In the commercial space, Java is the most used application language, people are diverse in their abilities, and they cost a lot. Worse yet, if they don't like their situation, they quit and move. And projects have far shorter lifespans -- if the ROI doesn't meet the hurdle rate, the project is killed.

These thoughts were triggered by a New York Times article, Military Supercomputer Sets Record

"Solving that programming problem is important because in just a few years personal computers will have microprocessor chips with dozens or even hundreds of processor cores. The industry is now hunting for new techniques for making use of the new computing power. Some experts, however, are skeptical that the most powerful supercomputers will provide useful examples."

Maybe that skepticism is justified, what do you think??

Explaining our secret sauce - Part Two

steveh — Fri, 6 Jun 2008 16:04:45 -0500

The first time I blogged on this, Explaining our secret sauce, I referenced the Wikipedia entry for "dataflow programming".

I have subsequently learned that it would be more accurate to point to Kahn Process Networks as the specific type of dataflow that was the genesis for our Pervasive DataRush library and engine.

"KPN is a common model for describing signal processing systems where infinite streams of data are incrementally transformed by processes executing in sequence or parallel. Despite parallel processes, multitasking or parallelism are not required for executing this model.

In a KPN, processes communicate via unbounded FIFO channels. Processes read and write atomic data elements or tokens from and to channels. Writing to a channel is non-blocking, i.e. it always succeeds and does not stall the process, while reading from a channel is blocking, i.e. a process that reads from an empty channel will stall and can only continue when the channel contains sufficient data items (tokens). Processes are not allowed to test an input channel for existence of tokens without consuming them. Given a specific input (token) history for a process, the process must be deterministic so that it always produces the same outputs (tokens). Timing or execution order of processes must not affect the result and therefore testing input channels for tokens is forbidden."

Multi-threaded development joins Gates as yesterday's man

steveh — Fri, 6 Jun 2008 11:26:51 -0500

"...the validity of multi threading is under attack. Veteran programmer Knuth said in a recent interview that multi threading may not be up to the task and could fail. As such, he is "unhappy" with the current trend towards multi-core architectures."

“To me, it looks more or less like the hardware designers have run out of ideas, and that they’re trying to pass the blame for the future demise of Moore’s Law to the software writers by giving us machines that work faster only on a few key benchmarks! I won’t be surprised at all if the whole multi threading idea turns out to be a flop..." Knuth said."

Analysis -- the "Plastics" of today

steveh — Thu, 5 Jun 2008 15:36:33 -0500

Before our current crop of graduates were born, there was the movie "The Graduate". One famous scene has our hero walking around at a party in his honor. He is given some sage career advice: "One word. Plastics".

The newest member of the DataRush team, who is still a student, found the following quote and thought it was interesting and very DataRush related. Hal Varian, Google’s chief economist (and intellectual superhero), says on a Freakonomics blog post:

“If you are looking for a career where your services will be in high demand, you should find something where you provide a scarce, complementary service to something that is getting ubiquitous and cheap. So what’s getting ubiquitous and cheap? Data. And what is complementary to data? Analysis. So my recommendation is to take lots of courses about how to manipulate and analyze data: databases, machine learning, econometrics, statistics, visualization, and so on.”

The X=X+1 Issue

steveh — Wed, 4 Jun 2008 16:03:15 -0500

A very clear explanation of why writing parallel apps is a challenge with procedural languages, versus declarative approaches. Although Java is procedural, our Java implementation of dataflow is avoids this pitfall.

"The ability to assign a memory location with variable data is the cornerstone of computer programming. It turns out the ability to re-assign that same memory location is perhaps one of the biggest consternations in the parallel programming world. At the time, multiple assignment seemed like (and was) a good idea. Coupled with a looping control structure, massive calculations could be performed. For instance, one could write a program to sum the numbers from 1 to 100 (and beyond with simple variable change):"

Let SUM = 0
FOR I = 1 to 100
LET SUM = SUM + I
NEXT

"While incredibly useful, multiple assignment creates “state” within the program because the value of variables can change as program runs. The typical programming language allows you to manage the state of program variables. i.e. the memory location that holds the variable SUM changes over time. As any programmer will tell you, program state is easily managed on a single Von Neuwman CPU through the plethora of programming languages. (Some may argue this point, however.) On a large number of CPUs, managing coupled program states becomes extremely difficult. In a parallel environment, it is up to the programmer to manage the state of I and SUM. In a sequential single CPU environment, the programmer can assume I = 100 after the loop completes. In a parallel environment, where the loop may have been broken into parts, I may equal 10 on one CPU, and 20 on another. Of course, the example is simple and obvious. Creating large complex parallel applications does not present such luxuries, however."

"The simple powerful idea of multiple assignment has come back to bite us in the asymptote."

Multicore is most disruptive

steveh — Mon, 2 Jun 2008 17:00:35 -0500

Gartner has published a list of the top 10 most disruptive technologies of the next 5 years, and multicore is number one.

You can see the whole article here

We would probably include cheap storage as the second item on the list - Fry's has 1-Terabyte drives for $169 this week - but the combination of cheap storage and multicore power is surely a disruptive pairing. Why is more power and space disruptive? It is because that additional capability is being provided in a new form, and this changes all the steps that come before it in the IT work flow. The old ways of utilizing this resource don't work, and the switch from a procedural code-path to a data flow architecture is certainly disruptive to all the programmers, analysts, coders, and architects that have to construct applications that run on these new platforms.

The whole industry seems to be waiting, holding back, looking for an approach that will save their efforts from the effects of this disruption. We encourage and are pushing for more examination and discussion around different approaches, because we believe that the more the software community understands the challenges, the better the dataflow paradigm looks for a whole category of applications. Is your team looking at multicore approaches?

Explaining our secret sauce

steveh — Fri, 30 May 2008 12:32:19 -0500

We wish we had some competition.

If there was some other vendor implementing dataflow, then we would not have the sole responsibility of teaching people about this approach. While there are certainly a large number of people who are familiar with the concept, most of the people we are meeting in presenting DataRush are not.

When we explain and illustrate the phenomenal results (as in 3 hours to 22 seconds), often the first question is "How can this be?" My response has been to point them to the Wikipedia entry for dataflow programming.

"Dataflow languages contrast with the majority of programming languages, which use the imperative programming model. In imperative programming the program is modeled as a series of operations, the data being effectively invisible. This distinction may seem minor, but the paradigm shift is fairly dramatic, and allows dataflow languages to be spread out across multicore, multiprocessor systems for free."

Check it out as a good place to start!

The tubas are blaring and the drums are pounding.

steveh — Fri, 30 May 2008 12:12:27 -0500

I just returned from our first presentation of DataRush to potential partners, SIs, and developers. While we have exhibited at a bunch of shows like JavaOne, Innotech, and the HP Global Showcase, and while we have briefed a bunch of smart press people and analysts, this was our first time with this type of audience.

Overall, we were very pleased at the interest and response to our story. The IT environment is looking for practical solutions to utilizing the multicore power that is flooding into the market, and these people see the tremendous opportunity inherent in platform transitions.

Today's Fry's ad shows a Gateway quad-core for $560.

Charles DeGaulle said that to be successful, one needs to "find a parade and get in front of it", and this parade is already marching down the street.

Follow-Up: I have attached the actual presentation I referred to above. Check it out!

Low-Latency Technology Outpacing Programmers’ Capabilities

steveh — Fri, 30 May 2008 11:45:02 -0500

"Again and again, executives said that finding enough programmers who are able to write "parallel" code -- programs that efficiently divide workloads across distributed processors -- is almost impossible. As Wall Street firms rely on multicore processing and even distributed computing to handle the ever-growing number of trade-related messages that are sensitive to any increase in data latency, the divergence between the capabilities of the technology and the capabilities of the programmers is becoming painfully evident.",

The Lawnmower Law

steveh — Wed, 21 May 2008 12:10:52 -0500

This article is a simple illustrative introduction to Amdahl's Law.

"Indeed, just as with parallel processors, there is a point of diminishing return. Adding the first 10 riding mowers reduced the time by 36 minutes. Adding another 30 only saved me 4 minutes. Adding 100 mowers makes little sense since I’ll never get below 20 minutes. (Although I would love to see such a lawn mowing demolition derby — in my neighbors yard of course.)"