pythondatabricksdatabricks-community-edition

DataBricks: notebook : Python: FileNotFoundError

I run the following code in DataBricks: notebook and get FileNotFoundError

import pandas as pd 
df = pd.read_csv ('E:\Myfolder1\Myfolder2\Myfolder3\myfile.csv')
print(df) 

FileNotFoundError: [Errno 2] No such file or directory: 'E:\\Myfolder1\\Myfolder2\\Myfolder3\\myfile.csv'

Why the change folder ' E:\Myfolder1\Myfolder2\Myfolder3\myfile.csv' to 'E:\Myfolder1\Myfolder2\Myfolder3\myfile.csv'

Solution

  • The error just is trying to show the string representation because \ character has a special meaning in Python & other languages - it's used as escape character for things like \n (new line), \t - tab, etc. For unknown combinations it's treated as plain \ that is represented correctly as \\.

    Regarding the actual error - you can't refer files on your local disk from the Databricks cluster that is running somewhere in the cloud. You need to upload this file to DBFS using UI or other ways (see docs), and then access it. Please note that Pandas can't work directly with files on DBFS and there is no /dbfs on community edition, so you need to follow recommendations of this answer to use dbutils to copy files locally.