Other Frameworks Integration

Posted April 24th, 2008 by porcelli

My product is based on Spring Framework, XMLBeans and iBatis. Is it possible integrate DataRush with these technologies?
If yes (I hope so)... do you have a path or samples for that?

Trackback URL for this post:

http://www.pervasivedatarush.com/trackback/207
‹ Shell script is not careful in finding DR_HOME Getting stack overflow error for join sample application ›
On May 6th, 2008 lyogman said:

The technologies you mention seem to imply that you're
working on a web app that manages a relational DB using transactional CRUD
operations. This isn't the nail that
DataRush is made to hammer. You might
want to read more about the purpose of DataRush here:

http://www.pervasivedatarush.com/node/194

Do you have a problem that involves batch processing of a large amount (e.g. 100 GB) of data?

 

 

On May 13th, 2008 Anonymous said:

Sorry, but it is not a web application. And I really have a batch processing of a large amount of data.
All my batch processing uses the technologies that I mentined. So.. is it possible integrate DataRush with that?

On May 15th, 2008 lyogman said:

From what I see here, ibates is an O/R mapper. We've never tried to integrate DataRush with such a tool. We are very wary of the often unpredictable and sometimes extremely poor performance of these tools, the SQL they generate, and the underlying RDBMS, particularly when queries are issued repeatedly for thousands or millions of rows

For efficiency, when we work with databases we try to get the information out and put it back in using single-shot bulk operations, staging the data as text. If we're looking for a little more convenience, we use the ReadFromJDBC and WriteToJDBC operators you'll find in the library. When configuring ReadFromJDBC, we try to select just the data we need, and when configuring WriteToJDBC we try to be smart about indexing and parallel table insert/update.

In between reading the data and writing it back, we generally process the data entirely outside the database. Databases are designed for transactional storage and indexed retrieval, not bulk processing.