News & views from the DataRush team

We are very fortunate to have a fairly long and interesting history of academic and commercial research on dataflow, flow-based programming, and other relevant topics.

A recent article in the New York Times discusses the many-core sea change that is taking place. The article is written with input from a few of Microsoft's researchers that are defining how Microsoft will respond to the many-core future. Interesting that the one scenario they chose to highlight in this very public article is this: analyzying your email to help you plan your day better.

After some interesting and insightful discussions with analysts and gurus, we have made a new presentation on DataRush. Please feel free to make comments or ask questions!

Hitachi has just announced new drive technology that will enable 4 terabytes of disk storage on your desktop within the next few years (1 terabyte on laptops). View the article here. More evidence that the data explosion will continue to accelerate. Now, combine that with the huge shift in processor technology towards multi-core processors. The result: more pressure for data processing applications to handle huge amounts of data taking advantage of tens and hundreds of cores. Massive parallel processing will have to go main stream. Give me a dual processor/quad-core machine with 4 terabytes of disk space and I'll have more processing and storage capacity on my desktop than many machines in data centers today. A bit of a stretch, but not by much.

What’s New in Pervasive DataRush Beta 2?

I used to be amazed at how little attention multi-core programming was getting.  But in the last year or two the attention factor has become exponential.  All the major chip manufactures are doing their best to educate the masses on how to write software on multiple core systems. Intel has started a multi-core programming training in India.  This kind of underscores what we have been saying from the beginning, to build

Java users know all of the advantages of using Java over C or C++. If you've forgotten, here are a few off the top of my head:

• An extremely rich set of libraries within the JDK and in the open source community
• Ease of programming and very strong IDE support
• Portability
• Native I/O speeds with the NIO package
• Huge industry investment in JVM technology
• Did I mention rich IDE support - once you've used NetBeans or Eclipse, who wants to go back to vi

But of course we always have the old argument of Java being too slow. For many years, that was a valid point. But no longer. Then along came JIT, NIO and GC improvements in release after release. Even in our benchmarking, we have seen several percentage increases in performance by moving to the latest version of the JVM.

Mike Hoskins, Pervasive CTO and General Manager of the Integration division, takes us back to the days when Pervasive engineers, tasked with developing a massively scaling hyperparallel data pump to run on abundantly inexpensive parallel hardware, debated in a common debate of the time: w