So I am importing a csv file that is about 2GB large with approximately 8 million regards using Toad and got the following error:
Exception of type 'System.OutOfMemoryException' was thrown.
at System.Text.RegularExpressions.RegexInterpreter.Goto(Int32 newpos)
at System.Text.RegularExpressions.RegexRunner.Scan(Regex regex, String text, Int32 textbeg, Int32 textend, Int32 textstart, Int32 prevlen, Boolean quick)
at System.Text.RegularExpressions.Regex.Run(Boolean quick, Int32 prevlen, String input, Int32 beginning, Int32 length, Int32 startat)
at Quest.Toad.Db.SqlFunction.EvaluateSql(Connection connection, String sql, Object parameters, IDataParameter& outparameters, Int32& inParameterIndex, Int32& outParameterIndex, Boolean function)
What do you think caused this?
What other alternatives are out there for a database of this magnitude?
I should note that mysql is hosted on windows.
you ran out of memory
break your csv up into chunks
I think that is the best practice moving forward to keep me from getting more gray hairs. I was ultimately able to do it using navicat.
My system has an i7 9300processor and 6GB DDR3 RAM. I think Navicat handled the memory much more efficiently.