pythonrefactoringautomated-refactoring

Automatically Update Python source code (imports)


We are refactoring our code base.

Old:

from a.b import foo_method

New:

from b.d import bar_method

Both methods (foo_method() and bar_method()) are the same. It just changed the name an the package.

Since above example is just one example of many ways a method can be imported, I don't think a simple regular expression can help here.

How to refactor the importing of a module with a command line tool?

A lot of source code lines need to be changed, so that an IDE does not help here.


Solution

  • Behind the scenes, IDEs are no much more than text editors with bunch of windows and attached binaries to make different kind of jobs, like compiling, debugging, tagging code, linting, etc. Eventually one of those libraries can be used to refactor code. One such library is Jedi, but there is one that was specifically made to handle refactoring, which is rope.

    pip3 install rope
    

    A CLI solution

    You can try using their API, but since you asked for a command line tool and there wasn't one, save the following file anywhere reachable (a known relative folder your user bin, etc) and make it executable chmod +x pyrename.py.

    #!/usr/bin/env python3
    from rope.base.project import Project
    from rope.refactor.rename import Rename
    from argparse import ArgumentParser
    
    def renamodule(old, new):
        prj.do(Rename(prj, prj.find_module(old)).get_changes(new))
    
    def renamethod(mod, old, new, instance=None):
        mod = prj.find_module(mod)
        modtxt = mod.read()
        pos, inst = -1, 0
        while True:
            pos = modtxt.find('def '+old+'(', pos+1)
            if pos < 0:
                if instance is None and prepos > 0:
                    pos = prepos+4 # instance=None and only one instance found
                    break
                print('found', inst, 'instances of method', old+',', ('tell which to rename by using an extra integer argument in the range 0..' if (instance is None) else 'could not use instance=')+str(inst-1))
                pos = -1
                break
            if (type(instance) is int) and inst == instance:
                pos += 4
                break # found
            if instance is None:
                if inst == 0:
                    prepos = pos
                else:
                    prepos = -1
            inst += 1
        if pos > 0:
            prj.do(Rename(prj, mod, pos).get_changes(new))
    
    argparser = ArgumentParser()
    #argparser.add_argument('moduleormethod', choices=['module', 'method'], help='choose between module or method')
    subparsers = argparser.add_subparsers()
    subparsermod = subparsers.add_parser('module', help='moduledottedpath newname')
    subparsermod.add_argument('moduledottedpath', help='old module full dotted path')
    subparsermod.add_argument('newname', help='new module name only')
    subparsermet = subparsers.add_parser('method', help='moduledottedpath oldname newname')
    subparsermet.add_argument('moduledottedpath', help='module full dotted path')
    subparsermet.add_argument('oldname', help='old method name')
    subparsermet.add_argument('newname', help='new method name')
    subparsermet.add_argument('instance', nargs='?', help='instance count')
    args = argparser.parse_args()
    if 'moduledottedpath' in args:
        prj = Project('.')
        if 'oldname' not in args:
            renamodule(args.moduledottedpath, args.newname)
        else:
            renamethod(args.moduledottedpath, args.oldname, args.newname)
    else:
        argparser.error('nothing to do, please choose module or method')
    

    Let's create a test environment with the exact the scenario shown in the question (here assuming a linux user):

    cd /some/folder/
    
    ls pyrename.py # we are in the same folder of the script
    
    # creating your test project equal to the question in prj child folder:
    mkdir prj; cd prj; cat << EOF >> main.py
    #!/usr/bin/env python3
    from a.b import foo_method
    
    foo_method()
    EOF
    mkdir a; touch a/__init__.py; cat << EOF >> a/b.py
    def foo_method():
        print('yesterday i was foo, tomorrow i will be bar')
    EOF
    chmod +x main.py
    
    # testing:
    ./main.py
    # yesterday i was foo, tomorrow i will be bar
    cat main.py
    cat a/b.py
    

    Now using the script for renaming modules and methods:

    # be sure that you are in the project root folder
    
    
    # rename package (here called module)
    ../pyrename.py module a b 
    # package folder 'a' renamed to 'b' and also all references
    
    
    # rename module
    ../pyrename.py module b.b d
    # 'b.b' (previous 'a.b') renamed to 'd' and also all references also
    # important - oldname is the full dotted path, new name is name only
    
    
    # rename method
    ../pyrename.py method b.d foo_method bar_method
    # 'foo_method' in package 'b.d' renamed to 'bar_method' and also all references
    # important - if there are more than one occurence of 'def foo_method(' in the file,
    #             it is necessary to add an extra argument telling which (zero-indexed) instance to use
    #             you will be warned if multiple instances are found and you don't include this extra argument
    
    
    # testing again:
    ./main.py
    # yesterday i was foo, tomorrow i will be bar
    cat main.py
    cat b/d.py
    

    This example did exact what the question did.

    Only renaming of modules and methods were implemented because it is the question scope. If you need more, you can increment the script or create a new one from scratch, learning from their documentation and from this script itself. For simplicity we are using current folder as the project folder, but you can add an extra parameter in the script to make it more flexible.