Stored Procedure or Code

Question

I am not asking for opinions but more on documentations.

We have a lot of data files (XML, CSV, Plantext, etc...), and need to process them, data mine them.

The lead database person suggested using stored procedure to accomplish the task. Basically we have a staging table where the file get serialized, and saved into a clob, or XML column. Then from there he suggested to further use stored procedure to process the file.

I'm an application developer with db background, more so on application development, and I might be bias, but using this logic in the DB seems like a bad idea and I am unable to find any documentation to prove or disapprove what I refer to as putting a car on a train track to pull a load of freights.

So my questions are: How well does the DB (Oracle, DB2, MySQL, SqlServer) perform when we talking about regular expression search, search and replace of data in a clob, dom traversal, recursion? In comparison to a programming language such as Java, PHP, or C# on the same issues.

Edit

So what I am looking for is documentation on comparison / runtime analysis of a particular programming language compare to a DBMS, in particular for string search and replace, regular expression search and replace. XML Dom traversal. Memory usage on recursive method calls. And in particular how well they scale when encountered with 10 - 100's of GB of data.

Answer 1

It sounds like you are going to throw business logic into the storage layer. For operations like you describe, you should not use the database. You may end up in trying to find workarounds for showstoppers or create quirky solutions because of inflexibility.

Also keep maintainability in mind. How many people will later be able to maintain the solution?

Speaking about speed, choosing the right programming language you will be able to process data in multiple threads. At the end, your feeling with the car n the train is right;)