pythonpandasregex

How to match a substring using a pattern and replace by passing a variable in RegEx, Python

I am trying to iterate through a Pandas dataframe's column values one by one to detect a substring with a RegEx pattern and replace it wherever it shows up. The string values in the dataframe's target column have the following variations:

  1. 'func(something)...'
  2. 'func(something,something)...'

I am attempting to target the values with the 2nd variation and replace the comma-separated substring by a value from another column in the same row.

My sample code looks like:

import re
for i, x in enumerate(df['col1']):
  df.loc[i,'col2'] = re.sub(r'^(func\().+,.+(\).*)', r'\1%s\2'%(x), df.loc[i,'col2'])

I have had a successful output on a terminal when I replace the * in the pattern with +:

>>> val = 'func(y,z) and func2(a,b)'
>>> val
'func(y,z) and func2(a,b)'
>>> pattern1 = r'^(func\().+,.+(\).*)'
>>> replacement = r'\1%s\2'%('x')
>>> val2 = re.sub(pattern1, replacement, val) 
>>> val2
'func(x)'
>>> pattern2 = r'^(func\().+,.+(\).+)'
>>> val2 = re.sub(pattern2, replacement, val) 
>>> val2
'func(x) and func2(a,b)'
>>> val3 = 'func(y) and func2(a,b)'
>>> val4 = re.sub(pattern2, replacement, val3) 
>>> val4
'func(y) and func2(a, b)'

I thought the .* in pattern1 should have accommodated for 0 or more instances of any character after the final captured parenthesis \), but at least using .+ in pattern2 works.

Problem is, when I try to replicate it on VSCode with Python on the dataframe's column values, I am not getting the desired output. I think my regex pattern is catching other parts of the strings I did not intend, so I am getting mixed results.

Am I missing anything with pattern2?

Update: Here is an example code I used in VSCode in which the result is promising, but when the string has more conditions added the pattern doesn't hold up.

df = pd.DataFrame({'col1':['func(a,b) and func(c,d)','func(a) and func(c)','func(b) and func(c,d)','func(a,b,c) and func(d,e,f)'],
                   'col2':['e','b','a','g']})
df
>>>
    col1    col2
0   func(a,b) and func(c,d) e
1   func(a) and func(c) b
2   func(b) and func(c,d)   a
3   func(a,b,c) and func(d,e,f) g

import re
df['col3'] = ''
for i,x in enumerate(df['col2']):
    pattern = r'^(func\().+,.+(\).+)'
    replacement = fr'\1{x}\2'
    df.loc[i,'col3'] = re.sub(pattern,replacement,df.loc[i,'col1'])
    if df.loc[i,'col3'] != df.loc[i,'col1']:
        print(df.loc[i,'col1'],'--->',df.loc[i,'col3'])
>>>
func(a,b) and func(c,d) ---> func(e) and func(c,d)
func(a,b,c) and func(d,e,f) ---> func(g) and func(d,e,f)

The above is what I expected to see, but in the attempt below, you can see additional func() conditions get dropped:

df = pd.DataFrame({'col1':['func(a,b) and func(c,d) and func2(z)','func(a) and func(c) and func2(z)','func(b) and func(c,d) and func2(z)','func(a,b,c) and func(d,e,f) and func2(z)'],
                   'col2':['e','b','a','g']})
df
>>>
    col1    col2
0   func(a,b) and func(c,d) and func2(z)    e
1   func(a) and func(c) and func2(z)    b
2   func(b) and func(c,d) and func2(z)  a
3   func(a,b,c) and func(d,e,f) and func2(z)    g

import re
df['col3'] = ''
for i,x in enumerate(df['col2']):
    pattern = r'^(func\().+(?:,.+)+(\).+)'
    replacement = fr'\1{x}\2'
    df.loc[i,'col3'] = re.sub(pattern,replacement,df.loc[i,'col1'])
    if df.loc[i,'col3'] != df.loc[i,'col1']:
        print(df.loc[i,'col1'],'--->',df.loc[i,'col3'])

>>>
func(a,b) and func(c,d) and func2(z) ---> func(e) and func2(z)
func(b) and func(c,d) and func2(z) ---> func(a) and func2(z)
func(a,b,c) and func(d,e,f) and func2(z) ---> func(g) and func2(z)

The pattern seems to leave out the middle func() expressions.

Solution

  • Prepare the replacement string for each iteration before inserting it in the re.sub(pattern, replacement, text_string) function.

    Updated based on new information on test string and desired output.

    REGEX PATTERN (Python re module):

    ^func\([^),]+(,[^,)]+)+\)

    Regex demo: https://regex101.com/r/aKyQSs/4

    PYTHON

    import re

df = {
    'col1':['func(a,b) and func(c,d)','func(a) and func(c)','func(b) and func(c,d)','func(a,b,c) and func(d,e,f)'], 
    'col2':['e','b','a','g']
}

df['col3'] = []

pattern = r'^func\([^),]+(,[^,)]+)+\)'

for i, x in enumerate(df['col2']): 
    replacement = r'func(%s)' % x
    result = re.sub(pattern, replacement, df['col1'][i])
    df['col3'].append((result))

print(df['col3'])

    OUTPUT:

    ['func(e) and func(c,d)', 'func(a) and func(c)', 'func(b) and func(c,d)', 'func(g) and func(d,e,f)']

    REGEX NOTES: