Data Import Tools - Large Database

Databases
1

So I am importing a csv file that is about 2GB large with approximately 8 million regards using Toad and got the following error:

System.OutOfMemoryException
Exception of type 'System.OutOfMemoryException' was thrown.
Stack Trace:
   at System.Text.RegularExpressions.RegexRunner.DoubleTrack()
   at System.Text.RegularExpressions.RegexRunner.EnsureStorage()
   at System.Text.RegularExpressions.RegexInterpreter.Goto(Int32 newpos)
   at System.Text.RegularExpressions.RegexInterpreter.Go()
   at System.Text.RegularExpressions.RegexRunner.Scan(Regex regex, String text, Int32 textbeg, Int32 textend, Int32 textstart, Int32 prevlen, Boolean quick)
   at System.Text.RegularExpressions.Regex.Run(Boolean quick, Int32 prevlen, String input, Int32 beginning, Int32 length, Int32 startat)
   at System.Text.RegularExpressions.Match.NextMatch()
   at Quest.Toad.Db.SqlFunction.EvaluateSql(Connection connection, String sql, Object[] parameters, IDataParameter[]& outparameters, Int32& inParameterIndex, Int32& outParameterIndex, Boolean function)

What do you think caused this?
What other alternatives are out there for a database of this magnitude?

2

I should note that mysql is hosted on windows.

3

you ran out of memory :slight_smile:

break your csv up into chunks

4

Good advice!

I think that is the best practice moving forward to keep me from getting more gray hairs. I was ultimately able to do it using navicat.
My system has an i7 9300processor and 6GB DDR3 RAM. I think Navicat handled the memory much more efficiently. :slight_smile: