excelvbatext

In VBA: Open, read, replace characters, and close text file in fastest possible way


I have a large text file (approx 1.7GB). I want to open it, read it into a string, replace various characters, write the string back to the text file and close it.

Is anyone able to advise how to make this faster?

My code works, but is pretty slow:

Function RemoveCharactersFromText(sFile as string) as string

Dim sText as String
dim sLineContect as String
Dim iFileNum as Integer

Open sFile for Input as iFileNum

Do Until EOF(iFileNum)
     Line Input #iFileNaum, sLineContent
     sText = sText & sLineContent & vbCrLf
Loop
Close iFileNum

On Error Resume Next
sText = Replace(sText, ",", "")
sText = Replace(sText, "#", "")
sText = Replace(sText, "*", "")
sText = Replace(sText, "+", "")
sText = Replace(sText, "=", "")
sText = Replace(sText, Chr(34), "")
sText = Replace(sText, "null", "")
On Error GoTo 0

iFileNum = FreeFile
Open sFile for Output as iFileNum
Print #iFileNum, sText
Close iFileNum

iFileNum = 0

I have tried writing the entire text file to the string in one hit:

Open sFile for Input as FreeFile
sText = Input(EOF(iFileNum), iFileNum)

I have also tried via FSO:

With CreateObject("Scripting.FileSystemObject")
     sText = .OpenTextFile(sFile, 1).ReadAll
End With

And get errors with both of these.

Any ideas? Thanks!

EDITED: I changed sTemp to sText. That was a typo when I was copying the code over. Sorry for the confusion!


Solution

  • My original post worked on 189MB but I could not read a 185 GB file using a buffer. This code processed the file in 121 seconds. Thanks GSerg.
    
    Sub Test2()
        Const SourceFilePath As String = "D:\Test.txt"
        Const DestFilePath As String = "D:\Test_Output.txt"
        Dim t As Double
        t = Timer
        ' Process the file
        RemoveTextInFileByReplace SourceFilePath, DestFilePath
        
        ' Replace the original file with the processed file
        On Error Resume Next
        Kill DestFilePath ' Delete the original source file
        On Error GoTo 0
        Debug.Print Timer - t
    End Sub
    
    Sub RemoveTextInFileByReplace(ByVal SourceFilePath As String, ByVal DestFilePath As String)
        Dim SourceFile As Integer, DestFile As Integer
        Dim FileLine As String
        
        ' Open the source file for reading
        SourceFile = FreeFile
        Open SourceFilePath For Input As #SourceFile
        
        ' Open the destination file for writing
        DestFile = FreeFile
        Open DestFilePath For Output As #DestFile
        
        ' Read the file line by line
        Do While Not EOF(SourceFile)
            Line Input #SourceFile, FileLine
            FileLine = Replace(FileLine, ",", "")
            FileLine = Replace(FileLine, "#", "")
            FileLine = Replace(FileLine, "*", "")
            FileLine = Replace(FileLine, "+", "")
            FileLine = Replace(FileLine, "=", "")
            FileLine = Replace(FileLine, Chr(34), "")
            FileLine = Replace(FileLine, "null", "")
            Print #DestFile, FileLine ' Write the modified line to the destination file
        Loop
        
        ' Close the files
        Close #SourceFile
        Close #DestFile
    End Sub