excelvbadata-processing

VBA code for processing data in an Excel file crashes after processing about 400-500 rows


I have coded a VBA macro to process downloaded data. The data has some junk rows that need to be deleted, but also has some rows where the data is off by a couple of columns and a couple of rows. The macro moves data to the right place and deletes unnecessary rows.

It works well for about 400-500 rows and then it crashes. The data I am trying to process is about 900-1,000 rows.

Here is an example of a few rows of the raw data:

Team 1 Batters
Schedule Rankings Trends Contracts Stats
Pos Players 13-May 5/14- 5/21 Proj Actual Rost Start Salary Contract * EXT BA R HR RBI SB
C Patrick Bailey C SF 213 195 0.17 0.05 5 E-2025 0.28 10 3 11 1
Not in Lineup
LAD,  9:45pm ETvs TBA Home: 5 (LAD,COL)Away: 1 (@PIT) 213 195 17% 5% 5 E-2025 0.281 10 3 11 1
C Keibert Ruiz C WAS 104 487 0.27 0.14 5 E-2025 ** X 0.15 5 2 6 0
Game Postponed
No Game Home: 2 (MIN)Away: 6 (@CHW,@PHI) 104 487 27% 14% 5 E-2025 ** X 0.147 5 2 6 0
1B Christian Walker 1B ARI 41 32 1 0.98 30 E-2025 ** 0.26 30 8 28 1
Batting 4th
CIN,  9:40pm ETvs TBA Home: 5 (CIN,DET)Away: 2 (@LAD) 41 32 100% 98% 30 E-2025 ** 0.255 30 8 28 1
2B Kevin Newman 2B,3B,SS ARI CIN,  9:40pm ETvs TBA Home: 5 (CIN,DET)Away: 2 (@LAD) N/A 247 2% 1% 10 E-2026 0.253 12 2 10 0
Batting 9th
3B Patrick Wisdom 1B,3B,OF CHC @ATL,  7:20pm ETvs TBA Home: 5 (PIT,ATL)Away: 2 (@ATL) 275 246 11% 5% 7 E-2026 0.289 5 2 7 3
Not in Lineup
SS Ketel Marte 2B,SS ARI CIN,  9:40pm ETvs TBA Home: 5 (CIN,DET)Away: 2 (@LAD) 46 17 100% 99% 27 E-2025 0.287 31 9 25 1
Batting 2nd
MI Ozzie Albies 2B ATL 16 85 1 0.97 31 E-2025 ** 0.28 21 3 19 2
Batting 2nd
CHC,  7:20pm ETvs TBA Home: 6 (CHC,SD)Away: 1 (@CHC) 16 85 100% 97% 31 E-2025 ** 0.279 21 3 19 2
CI Freddie Freeman 1B LAD 5 41 1 1 36 E-2024 ** 0.3 29 4 25 1
Batting 3rd
@SF,  9:45pm ETvs TBA Home: 6 (CIN,ARI)Away: 2 (@SF) 5 41 100% 100% 36 E-2024 ** 0.295 29 4 25 1
OF Harrison Bader OF NYM 200 135 0.14 0.09 6 E-2026 0.27 16 1 10 6
Batting 7th
PHI,  7:10pm ETvs TBA Home: 1 (PHI)Away: 7 (@PHI,@MIA,@CLE) 200 135 14% 9% 6 E-2026 0.273 16 1 10 6
OF Brenton Doyle OF COL 188 33 0.64 0.48 7 E-2026 0.27 29 5 13 9
Batting 6th
@SD,  9:40pm ETvs TBA Home: 0 Away: 6 (@SD,@SF,@OAK) 188 33 64% 48% 7 E-2026 0.273 29 5 13 9
OF Jake Fraley OF CIN 136 126 0.39 0.24 8 E-2024 0.29 17 1 7 7
Not in Lineup
@ARI,  9:40pm ETvs TBA Home: 1 (SD)Away: 6 (@ARI,@LAD) 136 126 39% 24% 8 E-2024 0.286 17 1 7 7
OF Blake Perkins OF MIL 325 148 0.11 0.06 10 E-2026 0.21 20 4 14 3
Not in Lineup
PIT,  7:40pm ETvs TBA Home: 2 (PIT)Away: 5 (@HOU,@MIA) 325 148 11% 6% 10 E-2026 0.214 20 4 14 3
OF Seiya Suzuki OF CHC 54 133 0.98 0.86 20 E-2025 0.27 16 4 15 1
Batting 2nd
@ATL,  7:20pm ETvs TBA Home: 5 (PIT,ATL)Away: 2 (@ATL) 54 133 98% 86% 20 E-2025 0.27 16 4 15 1
U Luis Matos OF SF 259 210 0.31 0.2 5 E-2025 0.39 4 2 17 0
Batting 8th
LAD,  9:45pm ETvs TBA Home: 5 (LAD,COL)Away: 1 (@PIT) 259 210 31% 20% 5 E-2025 0.385 4 2 17 0
Injured
OF Nolan Jones OF COL 27 344 0.82 0.08 10 E-2025 0.17 11 1 7 2
Not in Lineup
@SD,  9:40pm ETvs TBA Home: 0 Away: 6 (@SD,@SF,@OAK) 27 344 82% 8% 10 E-2025 0.17 11 1 7 2
Minors
2B Eguy Rosario 2B,3B SD COL,  9:40pm ETvs TBA Home: 2 (COL)Away: 5 (@ATL,@CIN) 304 311 2% 0% 1 E-2026 0.25 4 3 6 0
3B Cam Collier 3B CIN @ARI,  9:40pm ETvs TBA Home: 1 (SD)Away: 6 (@ARI,@LAD) N/A N/A 7% 0% 5 E-2026 0 0 0 0 0
3B Kody Hoese 3B LAD @SF,  9:45pm ETvs TBA Home: 6 (CIN,ARI)Away: 2 (@SF) N/A N/A 1% 0% 5 E-2026 0 0 0 0 0
SS Aidan Miller SS PHI @NYM,  7:10pm ETvs TBA Home: 6 (NYM,WAS,TEX)Away: 1 (@NYM) N/A N/A 7% 0% 5 E-2026 0 0 0 0 0
OF Owen Caissie OF CHC @ATL,  7:20pm ETvs TBA Home: 5 (PIT,ATL)Away: 2 (@ATL) N/A N/A 9% 0% 5 E-2026 0 0 0 0 0
OF Alexander Ramirez OF NYM PHI,  7:10pm ETvs TBA Home: 1 (PHI)Away: 7 (@PHI,@MIA,@CLE) N/A N/A 2% 0% 5 E-2026 0 0 0 0 0
U Tyler Black 1B,3B MIL PIT,  7:40pm ETvs TBA Home: 2 (PIT)Away: 5 (@HOU,@MIA) N/A 431 17% 0% 5 E-2026 0.227 1 0 0 2
U Mark Vientos 3B NYM N/A 354 0.06 0.02 5 E-2025 0.33 3 1 4 0
Not in Lineup
PHI,  7:10pm ETvs TBA Home: 1 (PHI)Away: 7 (@PHI,@MIA,@CLE) N/A 354 6% 2% 5 E-2025 0.333 3 1 4 0
Team 1 Pitchers
Schedule Rankings Trends Contracts Stats
Pos Players 13-May 5/14- 5/21 Proj Actual Rost Start Salary Contract * EXT ERA WHIP W K S
P Kyle Harrison P SF LAD,  9:45pm ET Home: 5 (LAD,COL)Away: 1 (@PIT) 126 74 86% 48% 5 E-2026 3.6 1.27 4 49 0
P A.J. Minter P ATL CHC,  7:20pm ET Home: 6 (CHC,SD)Away: 1 (@CHC) 236 75 32% 24% 8 E-2026 3.57 1.02 5 20 1
P Hector Neris P CHC @ATL,  7:20pm ET Home: 5 (PIT,ATL)Away: 2 (@ATL) 225 93 56% 47% 5 E-2026 2.65 1.65 4 15 6
P Mitchell Parker P WAS No Game Home: 2 (MIN)Away: 6 (@CHW,@PHI) N/A 157 33% 18% 10 E-2026 3.09 1.12 2 25 0

Here is the desired output:

Team 1 Batters
Schedule Rankings Trends Contracts Stats
Pos Players 13-May 5/14- 5/21 Proj Actual Rost Start Salary Contract * EXT BA R HR RBI
C Patrick Bailey C SF 213 195 0.18 0.07 5 E-2025 0.281 10 3 11
C Keibert Ruiz C WAS 104 489 0.28 0.13 5 E-2025 ** X 0.147 5 2 6
1B Christian Walker 1B ARI 41 30 1 0.98 30 E-2025 ** 0.259 30 8 28
2B Kevin Newman 2B,3B,SS ARI CIN,  9:40pm ETvs TBA Home: 5 (CIN,DET)Away: 2 (@LAD) N/A 239 1% 1% 10 E-2026 0.253 12 2 10
3B Patrick Wisdom 1B,3B,OF CHC @ATL,  7:20pm ETvs TBA Home: 5 (PIT,ATL)Away: 2 (@ATL) 275 238 11% 6% 7 E-2026 0.294 5 2 7
SS Ketel Marte 2B,SS ARI CIN,  9:40pm ETvs TBA Home: 5 (CIN,DET)Away: 2 (@LAD) 46 20 100% 99% 27 E-2025 0.288 30 9 24
MI Ozzie Albies 2B ATL 16 84 1 0.97 31 E-2025 ** 0.279 21 3 19

Here is the code:

Option Explicit

Public Function ContainsArrayValue(StringToCheck As String, ValuesToLookFor As Variant) As Boolean
    Dim i As Long
    For i = LBound(ValuesToLookFor) To UBound(ValuesToLookFor)
        If InStr(1, StringToCheck, ValuesToLookFor(i)) > 0 Then
            ContainsArrayValue = True
            Exit Function
        End If
    Next i
    ContainsArrayValue = False
End Function

Public Function IsInArray(StringToBeFound As String, ValuesToCheck As Variant) As Boolean
    Dim i As Long
    For i = LBound(ValuesToCheck) To UBound(ValuesToCheck)
        If ValuesToCheck(i) = StringToBeFound Then
            IsInArray = True
            Exit Function
        End If
    Next i
    IsInArray = False
End Function

Public Function IsDivisible(x As Integer, d As Integer) As Boolean
    IsDivisible = (x Mod d) = 0
End Function

Sub CleanUpTwoLines(ReferenceRow As Range)
    Dim i As Long
    Dim TargetRange As Range
    Dim SourceRange As Range
    Dim TargetRow1 As Range
    Dim TargetRow2 As Range

    Set TargetRange = Range(ReferenceRow.Cells(1, 5), ReferenceRow.Cells(1, 17))
    Set SourceRange = TargetRange.Offset(2, -1)
    
    For i = 1 To 13
        TargetRange.Cells(1, i).Value = SourceRange.Cells(1, i)
    Next i

    Set TargetRow1 = ReferenceRow.Offset(1, 0).EntireRow
    Set TargetRow2 = ReferenceRow.Offset(2, 0).EntireRow
    
    Application.EnableEvents = False
    Application.ScreenUpdating = False
    TargetRow1.Delete
    TargetRow2.Delete
    Application.EnableEvents = True
    Application.ScreenUpdating = True
End Sub

Sub CleanUpOneLine(ReferenceRow As Range)
    Dim TargetRow1 As Range
    Set TargetRow1 = ReferenceRow.Offset(1, 0).EntireRow
    
    Application.EnableEvents = False
    Application.ScreenUpdating = False
    TargetRow1.Delete
    Application.EnableEvents = True
    Application.ScreenUpdating = True
End Sub

Sub PlayerRowCleanup(PlayerRow As Range, PlayerPosValues As Variant)
    If IsInArray(PlayerRow.Cells(1, 1).Offset(2, 0).Value, PlayerPosValues) = False Then                                    'We already know the line below is not a player; if the line two rows below is also not a player...
        Call CleanUpTwoLines(PlayerRow)                                                                                                         'Then do the two-line cleanup, which involves moving stats and deleting those two rows
    Else                                                                                                                                                            'If it's in fact just one row that's not a player...
        Call CleanUpOneLine(PlayerRow)                                                                                                           'Then delete that row
    End If
    
    While IsInArray(PlayerRow.Cells(1, 1).Offset(1, 0).Value, PlayerPosValues) = False                                     'Check if there are any extra non-player rows...
        PlayerRow.Cells(1, 1).Offset(1, 0).EntireRow.Delete                                                                               'If so delete them
    Wend
End Sub

Sub CleanUpAllRosterData()
    Dim JunkRowValues() As String
    Dim PlayerPosValues() As String
    Dim a As Range, b As Range
    Dim RowsToDelete As Range
    Dim counter As Integer

    JunkRowValues = Split("Batters,Pitchers,Minors,Injured,Pos,pitchers,Schedule,Not In Lineup", ",")
    PlayerPosValues = Split("C,1B,2B,SS,3B,MI,CI,OF,U,P", ",")
    
    Set a = Range("AllTeamsData")
    a.Cells.UnMerge
    counter = 0
    
    Application.ScreenUpdating = False
    Application.Calculation = xlCalculationManual
    
    For Each b In a.Rows
        counter = counter + 1
        If IsDivisible(counter, 20) Then
            If MsgBox("Analyzing row " & b.Row & " processed " & counter & " rows. Continue?", vbYesNo) = vbNo Then
                Exit For
            End If
        End If
        
        If ContainsArrayValue(b.Cells(1, 1).Value, JunkRowValues) Then                                                        'Check whether the current row is junk
            If RowsToDelete Is Nothing Then                                                                                                  'If it is delete it
                Set RowsToDelete = b.EntireRow
            Else
                Set RowsToDelete = Union(RowsToDelete, b.EntireRow)
            End If
        ElseIf IsInArray(b.Cells(1, 1).Value, PlayerPosValues) Then                                                                 'Check whether the current row is a player row
            If Not IsInArray(b.Cells(1, 1).Offset(1, 0).Value, PlayerPosValues) Then                                         'If the next line is also a player row, then no need to do anything--go to the next row
                Call PlayerRowCleanup(b, PlayerPosValues)                                                                                'If the next line is not also a player row, then call for cleanup
            End If
        End If
    Next b
    
    If Not RowsToDelete Is Nothing Then
        RowsToDelete.Delete
    End If
    
    Application.ScreenUpdating = True
    Application.Calculation = xlCalculationAutomatic
End Sub

I would greatly appreciate any thoughts on what is causing the crash after the first 400-500 rows.

I have run the code through ChatGPT and followed its suggestions. The macro still crashes.


Solution

  • Note:

    Microsoft documentation:

    InStr function

    Range.Resize property (Excel)

    Range.AutoFit method (Excel)

    Option Explicit
    Sub Demo()
        Dim i As Long, j As Long
        Dim arrData, rngData As Range, sErrList As String
        Dim arrRes, iR As Long, iMode As Long
        Dim oSht As Worksheet, RowCnt As Long
         ' modify as needed, the 1st and last char should be "|"
        Const POS = "|C|1B|2B|SS|3B|MI|CI|OF|U|P|"
        Const SKIP_ROWS = 3
        Set oSht = Sheets("Data") ' modify as needed
        ' load data into an array
        arrData = oSht.UsedRange.Value
        arrRes = arrData
        RowCnt = UBound(arrData)
        iR = SKIP_ROWS
        ' loop through data
        For i = LBound(arrData) + SKIP_ROWS To RowCnt
            If Len(arrData(i, 1)) > 0 And InStr(1, POS, "|" & arrData(i, 1) & "|", vbTextCompare) > 0 Then
                ' get the row count for each player
                If RowCnt - i < 3 Then
                    iMode = RowCnt - i + 1
                ElseIf InStr(1, POS, "|" & arrData(i + 3, 1) & "|", vbTextCompare) Then
                    iMode = 3
                ElseIf InStr(1, POS, "|" & arrData(i + 2, 1) & "|", vbTextCompare) Then
                    iMode = 2
                Else
                    iMode = 0
                End If
                Select Case iMode
                Case 0 ' missing data
                    sErrList = sErrList & "," & arrData(i, 1)
                Case 2 ' two rows
                    iR = iR + 1
                    For j = LBound(arrData, 2) To UBound(arrData, 2)
                        arrRes(iR, j) = arrData(i, j)
                    Next j
                    i = i + 1
                Case 3 ' three rows
                    iR = iR + 1
                    For j = LBound(arrData, 2) To UBound(arrData, 2)
                        If j < 3 Then
                            arrRes(iR, j) = arrData(i, j)
                        Else
                            arrRes(iR, j) = arrData(i + 2, j - 1)
                        End If
                    Next j
                    i = i + 2
                End Select
            End If
        Next i
        ' write output to sheet
        Sheets.Add
        Range("A2").Resize(iR, UBound(arrData, 2)).Value = arrRes
        ActiveSheet.UsedRange.EntireColumn.AutoFit
        Columns(1).ColumnWidth = 4
        If Len(sErrList) > 0 Then
            MsgBox "Missing data for:" & vbCr & Mid(sErrList, 2)
        Else
            MsgBox "Done"
        End If
    End Sub
    

    enter image description here


    Update:

    Option Explicit
    Sub Demo()
        Dim i As Long, j As Long
        Dim arrData, rngData As Range, sErrList As String
        Dim arrRes, iR As Long, iMode As Long, iNext As Long
        Dim oSht As Worksheet, RowCnt As Long, bHeader As Boolean
         ' modify as needed, the 1st and last char should be "|"
        Const POS = "|C|1B|2B|SS|3B|MI|CI|OF|U|P|"
        Const SKIP_ROWS = 3
        Set oSht = Sheets("Data") ' modify as needed
        ' load data into an array
        arrData = oSht.UsedRange.Value
        arrRes = arrData
        RowCnt = UBound(arrData)
        iR = SKIP_ROWS
        ' loop through data
        For i = LBound(arrData) + SKIP_ROWS To RowCnt
            If Len(arrData(i, 1)) > 0 And InStr(1, POS, "|" & arrData(i, 1) & "|", vbTextCompare) > 0 Then
                ' locate the next player
                iMode = 0: iNext = 0: bHeader = False
                For j = i + 1 To RowCnt
                    If InStr(1, POS, "|" & arrData(j, 1) & "|", vbTextCompare) Then
                        iNext = j ' start row# of next player
                        Exit For
                    ElseIf UCase(Left(arrData(j, 1), 4)) = "TEAM" Then
                        bHeader = True ' extra header
                    End If
                Next
                ' get the row count for each player
                If iNext > 0 Then
                    iMode = iNext - i
                    If bHeader Then iMode = iMode - SKIP_ROWS
                Else
                    If RowCnt - i < 3 Then
                        iMode = RowCnt - i + 1
                    End If
                End If
                Select Case iMode
                Case 0 ' missing data
                    sErrList = sErrList & "," & arrData(i, 2)
                Case 1, 2 ' two rows
                    iR = iR + 1
                    For j = LBound(arrData, 2) To UBound(arrData, 2)
                        arrRes(iR, j) = arrData(i, j)
                    Next j
                    i = i + (iMode - 1)
                Case 3, 4 ' three rows
                    iR = iR + 1
                    For j = LBound(arrData, 2) To UBound(arrData, 2)
                        If j < 3 Then
                            arrRes(iR, j) = arrData(i, j)
                        Else
                            arrRes(iR, j) = arrData(i + 2, j - 1)
                        End If
                    Next j
                    i = i + 2
                End Select
                If bHeader Then i = i + SKIP_ROWS ' skip extra header
            End If
        Next i
        ' write output to sheet
        Sheets.Add
        Range("A2").Resize(iR, UBound(arrData, 2)).Value = arrRes
        ActiveSheet.UsedRange.EntireColumn.AutoFit
        Columns(1).ColumnWidth = 4
        If Len(sErrList) > 0 Then
            MsgBox "Missing data for:" & vbCr & Mid(sErrList, 2)
        Else
            MsgBox "Done"
        End If
    End Sub
    

    enter image description here