pythonpython-3.xshelve

How do I prevent shelve module from appending ".db" to filename?


Python: 3.12.2, OS: MacOS 13.6.6

When I specify a filename to shelve.open, it appends .db to the filename:

% ls

% python
Python 3.12.2 (main, Feb  6 2024, 20:19:44) [Clang 15.0.0 (clang-1500.1.0.2.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import shelve
>>> with shelve.open("myfile"):
...     pass
... 
>>> quit()

% ls
myfile.db

Furthermore, if I attempt to open an existing file as "myfile.db" (with the extension), I get the following error:

% python
Python 3.12.2 (main, Feb  6 2024, 20:19:44) [Clang 15.0.0 (clang-1500.1.0.2.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import shelve
>>> with shelve.open("myfile.db"):
...     pass
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/shelve.py", line 243, in open
    return DbfilenameShelf(filename, flag, protocol, writeback)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/shelve.py", line 227, in __init__
    Shelf.__init__(self, dbm.open(filename, flag), protocol, writeback)
                         ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/dbm/__init__.py", line 89, in open
    raise error[0]("db type could not be determined")
dbm.error: db type could not be determined

Opening the existing file as simply "myfile" with no extension works fine, however.

  1. How do I prevent shelve.open from appending ".db" to the filename?
  2. Why can't I open existing databases if I specify their ".db" extension?

Neither of these issues happen on Python 3.10.12 on Ubuntu 22, so I'm not sure if it's a Python version thing, or a platform thing.


Solution

  • Why can't I open existing databases if I specify their ".db" extension?

    After scrying shelve source code it could be unveiled that shelve.open("myfile") does result in calling dbm.open(filename, 'c') After scrying dbm source code it could be unveiled that this does depend on dbm.whichdb, where following line could be found

    f = io.open(filename + b".db", "rb")
    

    therefore if you attempt to do shelve.open("myfile.db") whichdb would be looking for myfile.db.db.

    How do I prevent shelve module from appending ".db" to filename?

    If you would do that this would most likely cause dbm.whichdb to malfunction for ndbm and dumbdm as it rely on extensions to detect nature of system used, as unveiled in comments

    # Check for ndbm first -- this has a .pag and a .dir file
    # Check for dumbdbm next -- this has a .dir and a .dat file
    

    whilst for sqlite3 and gnu it should would work independently from extensions as detection rely on headers (see source code linked above for details).