I have a large text file (approx 1.7GB). I want to open it, read it into a string, replace various characters, write the string back to the text file and close it.
Is anyone able to advise how to make this faster?
My code works, but is pretty slow:
Function RemoveCharactersFromText(sFile as string) as string
Dim sText as String
dim sLineContect as String
Dim iFileNum as Integer
Open sFile for Input as iFileNum
Do Until EOF(iFileNum)
Line Input #iFileNaum, sLineContent
sText = sText & sLineContent & vbCrLf
Loop
Close iFileNum
On Error Resume Next
sText = Replace(sText, ",", "")
sText = Replace(sText, "#", "")
sText = Replace(sText, "*", "")
sText = Replace(sText, "+", "")
sText = Replace(sText, "=", "")
sText = Replace(sText, Chr(34), "")
sText = Replace(sText, "null", "")
On Error GoTo 0
iFileNum = FreeFile
Open sFile for Output as iFileNum
Print #iFileNum, sText
Close iFileNum
iFileNum = 0
I have tried writing the entire text file to the string in one hit:
Open sFile for Input as FreeFile
sText = Input(EOF(iFileNum), iFileNum)
I have also tried via FSO:
With CreateObject("Scripting.FileSystemObject")
sText = .OpenTextFile(sFile, 1).ReadAll
End With
And get errors with both of these.
Any ideas? Thanks!
EDITED: I changed sTemp to sText. That was a typo when I was copying the code over. Sorry for the confusion!
My original post worked on 189MB but I could not read a 185 GB file using a buffer. This code processed the file in 121 seconds. Thanks GSerg.
Sub Test2()
Const SourceFilePath As String = "D:\Test.txt"
Const DestFilePath As String = "D:\Test_Output.txt"
Dim t As Double
t = Timer
' Process the file
RemoveTextInFileByReplace SourceFilePath, DestFilePath
' Replace the original file with the processed file
On Error Resume Next
Kill DestFilePath ' Delete the original source file
On Error GoTo 0
Debug.Print Timer - t
End Sub
Sub RemoveTextInFileByReplace(ByVal SourceFilePath As String, ByVal DestFilePath As String)
Dim SourceFile As Integer, DestFile As Integer
Dim FileLine As String
' Open the source file for reading
SourceFile = FreeFile
Open SourceFilePath For Input As #SourceFile
' Open the destination file for writing
DestFile = FreeFile
Open DestFilePath For Output As #DestFile
' Read the file line by line
Do While Not EOF(SourceFile)
Line Input #SourceFile, FileLine
FileLine = Replace(FileLine, ",", "")
FileLine = Replace(FileLine, "#", "")
FileLine = Replace(FileLine, "*", "")
FileLine = Replace(FileLine, "+", "")
FileLine = Replace(FileLine, "=", "")
FileLine = Replace(FileLine, Chr(34), "")
FileLine = Replace(FileLine, "null", "")
Print #DestFile, FileLine ' Write the modified line to the destination file
Loop
' Close the files
Close #SourceFile
Close #DestFile
End Sub