Python - Pyramid and matplotlib - Cannot Have More Than One View Output A SVG?

I am developing a Python Pyramid application where I am intending to create more than one SVG image to chart statistics using pie charts. In my testing I find that one SVG view works correctly and as soon as I add a second SVG output view, and a second SVG image is loaded (order of SVG image load doesn't matter), whether directly through its view, or through another view that references this view, the SVG images "are combined" in any other further calls to load a SVG file. This appears to be a bug somewhere in the Python stack as it appears memory is not cleared properly (primarily in the case of more than one SVG file, see further details below). Also note below that after enough image/page loads a TclError is encountered.

Since I was using SVG in a more detailed application with many more views, I am reproducing this in a minimized/reduced application to show it isn't something extra I'm doing and this code is generated right from the Pyramid alchemy template and database calls are not involved. The database is actively utilized in my more details application. This application only has 3 views, where the first view is part of the original template. I am also adding DEBUG logging to make it clear that there is no indication that there is any internal calling of the other SVG view.

Some of the view code is based on Matplotlib svg as string and not a file primarily for the use of StringIO. Note that as a pie chart is needed, that is the primary reason why my code differs from the code in referenced question. I find the issue is essentially the same whether I use StringIO or cStringIO. In my code I am using cStringIO.

The full application code is available at: https://github.com/danielpronych/pyramidapp

PyPlot Documentation:: pyplot api

Update: Code From First SVG View (figure handles and a close command):

@view_config(route_name='view_test_svg')
def test_svg_view(request):
    # Full module import is not allowed by Pyramid
    #from pylab import *
    # Do individual required imports instead
    from pylab import close, figure, axes, pie, title, savefig
    # For clarity, note that the above, and below, function the same
    #from matplotlib.pyplot import close, figure, axes, pie, title, savefig
    log.debug('In test_svg_view')
    fig = figure(1, figsize=(6,6))
    ax = axes([0.1, 0.1, 0.8, 0.8])
    labels = ['Frogs', 'Hogs', 'Dogs', 'Logs']
    fracs = [15, 30, 45, 10]
    explode=(0, 0.05, 0, 0)
    pie(fracs, explode=explode, labels=labels,
                                autopct='%1.1f%%', shadow=True, startangle=90)
    title('Raining Hogs and Dogs', bbox={'facecolor':'0.8', 'pad':5})
    imgdata = cStringIO.StringIO()
    fig.savefig(imgdata, format='svg')
    imgdata.seek(0)
    svg_dta = imgdata.getvalue()
    # Close the StringIO buffer
    imgdata.close()
    close('all')
    return Response(svg_dta, content_type='image/svg+xml')

Steps For This Code:

Load http://localhost:6543/test.svg
Load http://localhost:6543/test2.svg
Load either (1) or (2) again. The pserve will "spin" continuously with no sign from the command window why.

Note: Loading the same SVG view 3 times with this code also has the same result as the steps above which does not occur with the code below.

Also note that pyplot pie does have a return; however, not a "handle" per se.

pserve window does not respond to Control+C commands, even after waiting over a minute (particularly after doing the steps above).

Code From First SVG View (original, steps below are from this code):

@view_config(route_name='view_test_svg')
def test_svg_view(request):
    # Full module import is not allowed by Pyramid
    #from pylab import *
    # Do individual required imports instead
    from pylab import figure, axes, pie, title, savefig
    # For clarity, note that the above, and below, function the same
    #from matplotlib.pyplot import figure, axes, pie, title, savefig
    log.debug('In test_svg_view')
    figure(1, figsize=(6,6))
    ax = axes([0.1, 0.1, 0.8, 0.8])
    labels = ['Frogs', 'Hogs', 'Dogs', 'Logs']
    fracs = [15, 30, 45, 10]
    explode=(0, 0.05, 0, 0)
    pie(fracs, explode=explode, labels=labels,
                                autopct='%1.1f%%', shadow=True, startangle=90)
    title('Raining Hogs and Dogs', bbox={'facecolor':'0.8', 'pad':5})
    imgdata = cStringIO.StringIO()
    savefig(imgdata, format='svg')
    imgdata.seek(0)
    svg_dta = imgdata.getvalue()
    # Close the StringIO buffer
    imgdata.close()
    return Response(svg_dta, content_type='image/svg+xml')

Python Version: Python 2.7.5

Python Package Configuration (Primary Packages Only)

pyramid-1.6a1-py2.7
matplotlib-1.4.3-py2.7-win32

Steps Taken To Reproduce:

pserve pyramidapp.

Command: pserve development.ini --reload

Starting server in PID 4912.
serving on http://0.0.0.0:6543

Load http://localhost:6543/test.svg

Note this works properly

DEBUG [pyramidapp.views:22][Dummy-2] In test_svg_view

Step 2 image

Load http://localhost:6543/test2.svg

Note this "combines" both SVG files together

DEBUG [pyramidapp.views:45][Dummy-3] In test2_svg_view

Step 3 image

Load http://localhost:6543/test.svg

Note this works exactly like test2.svg, with the correct title, since they are also of similar length, and now images are combined in this view as well

DEBUG [pyramidapp.views:22][Dummy-4] In test_svg_view

Step 4 image

Rehost application and only load http://localhost:6543/test2.svg

Note this works properly for first load as this view was loaded before test.svg this time

DEBUG [pyramidapp.views:45][Dummy-2] In test2_svg_view

Step 5 image

Tracelog when using Control+C to terminate the pserve process

Error in sys.exitfunc:
Traceback (most recent call last):
  File "--python_path--\lib\atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "--python_path--\lib\site-packages\matplotlib-1.4.3-py2.7-win32.egg\ma
tplotlib\_pylab_helpers.py", line 89, in destroy_all
    manager.destroy()
  File "--python_path--\lib\site-packages\matplotlib-1.4.3-py2.7-win32.egg\ma
tplotlib\backends\backend_tkagg.py", line 588, in destroy
    self.window.destroy()
  File "--python_path--\lib\lib-tk\Tkinter.py", line 1789, in destroy
    for c in self.children.values(): c.destroy()
  File "--python_path--\lib\lib-tk\Tkinter.py", line 2042, in destroy
    self.tk.call('destroy', self._w)
_tkinter.TclError: out of stack space (infinite loop?)
^C caught in monitor process

Important: After enough SVG image loads the following is encountered:

The only way to fix this currently is to restart pserve. Also note that views, such as the my_view load properly as long as SVG images are not referenced, or utilized, by such views.

Another important note, as long as only one SVG file, i.e. http://localhost:6543/test.svg, is loaded the entire time of pserve it seems that image can be reloaded/refreshed (potentially) infinite times without any apparent issue, or encountering of the following:

_tkinter header

_tkinter.TclError
TclError: out of stack space (infinite loop?)
Traceback (most recent call last)
File "--python_path--\lib\site-packages\pyramid_debugtoolbar-2.0.2-py2.7.egg\pyramid_debugtoolbar\panels

\performance.py", line 69, in noresource_timer_handler
Display the sourcecode for this frameOpen an interactive python shell in this frameresult = handler(request)
File "--python_path--\lib\site-packages\pyramid-1.6a1-py2.7.egg\pyramid\tweens.py", line 20, in excview_tween
Display the sourcecode for this frameOpen an interactive python shell in this frameresponse = handler(request)
File "--python_path--\lib\site-packages\pyramid_tm-0.11-py2.7.egg\pyramid_tm\__init__.py", line 94, in tm_tween
Display the sourcecode for this frameOpen an interactive python shell in this framereraise(*exc_info)
File "--python_path--\lib\site-packages\pyramid_tm-0.11-py2.7.egg\pyramid_tm\__init__.py", line 75, in tm_tween
Display the sourcecode for this frameOpen an interactive python shell in this frameresponse = handler(request)
File "--python_path--\lib\site-packages\pyramid-1.6a1-py2.7.egg\pyramid\router.py", line 145, in handle_request
Display the sourcecode for this frameOpen an interactive python shell in this frameview_name
File "--python_path--\lib\site-packages\pyramid-1.6a1-py2.7.egg\pyramid\view.py", line 527, in _call_view
Display the sourcecode for this frameOpen an interactive python shell in this frameresponse = view_callable

(context, request)
File "--python_path--\lib\site-packages\pyramid-1.6a1-py2.7.egg\pyramid\config\views.py", line 384, in 

viewresult_to_response
Display the sourcecode for this frameOpen an interactive python shell in this frameresult = view(context, 

request)
File "--python_path--\lib\site-packages\pyramid-1.6a1-py2.7.egg\pyramid\config\views.py", line 506, in 

_requestonly_view
Display the sourcecode for this frameOpen an interactive python shell in this frameresponse = view(request)
File "c:\projects\python\pyramid\pyramidapp\pyramidapp\views.py", line 55, in test2_svg_view
Display the sourcecode for this frameOpen an interactive python shell in this framesavefig(imgdata, 

format='svg')
File "--python_path--\lib\site-packages\matplotlib-1.4.3-py2.7-win32.egg\matplotlib\pyplot.py", line 578, in 

savefig
Display the sourcecode for this frameOpen an interactive python shell in this framedraw()   # need this if 

'transparent=True' to reset colors
File "--python_path--\lib\site-packages\matplotlib-1.4.3-py2.7-win32.egg\matplotlib\pyplot.py", line 571, in 

draw
Display the sourcecode for this frameOpen an interactive python shell in this frameget_current_fig_manager

().canvas.draw()
File "--python_path--\lib\site-packages\matplotlib-1.4.3-py2.7-win32.egg\matplotlib\backends\backend_tkagg.py", 

line 350, in draw
Display the sourcecode for this frameOpen an interactive python shell in this frametkagg.blit(self._tkphoto, 

self.renderer._renderer, colormode=2)
File "--python_path--\lib\site-packages\matplotlib-1.4.3-py2.7-win32.egg\matplotlib\backends\tkagg.py", line 

24, in blit
Display the sourcecode for this frameOpen an interactive python shell in this frametk.call("PyAggImagePhoto", 

photoimage, id(aggimage), colormode, id(bbox_array))
TclError: out of stack space (infinite loop?)

Update (September 2015): I was hoping to see a further update from Sergey, since his suggested solution did seem to help at first; however, the issue was not resolved with his solution, since none has happened even though I've waited a considerable amount of time, I found that only my solution actually resolved this issue. Lastly, to be clear, my solution works in the web-based implementation and in batch processing as this issue surfaces in batch processing as well.

Solution

The answer provided by Sergey was definitely helpful. I was able to find that if added a few more pieces to the code, based on portions of this example, including a matplotlib.use statement, the memory issues seem to be resolved. I tested this both in my test application (2 views generating a SVG file, posted above) and my primary application which currently has 3 SVG views, which I will expand to more views creating SVG files.

With my primary application, I can refresh between all 3 existing views, multiple times, without the SVG image combining that I found with prior versions of this code. The setting seems necessary for more concurrent uses which may be why other SO questions don't seem to reflect this issue, unless I am able to find something else that is causing this issue. I am not sure if this is the best answer; however, this does appear to work correctly for now and I plan to monitor to see if any other issues are found.

@view_config(route_name='view_test_svg')
def test_svg_view(request):
    from matplotlib import use as matplotlib_use
    matplotlib_use("Svg")
    from matplotlib.pyplot import close, figure, axes, pie, title, savefig
    log.debug('In test_svg_view')
    fig = figure(1, figsize=(6,6))
    ax = fig.add_axes([0.1, 0.1, 0.8, 0.8])
    labels = ['Frogs', 'Hogs', 'Dogs', 'Logs']
    fracs = [15, 30, 45, 10]
    explode=(0, 0.05, 0, 0)
    pie(fracs, explode=explode, labels=labels,
                                autopct='%1.1f%%', shadow=True, startangle=90)
    title('Raining Hogs and Dogs', bbox={'facecolor':'0.8', 'pad':5})
    imgdata = cStringIO.StringIO()
    savefig(imgdata, format='svg')
    imgdata.seek(0)
    svg_dta = imgdata.getvalue()
    # Close the StringIO buffer
    imgdata.close()
    close('all')
    return Response(svg_dta, content_type='image/svg+xml')