I'm trying to run a script in the pyspark environment but so far I haven't been able to.
How can I run a script like python script.py but in pyspark?
You can do: ./bin/spark-submit mypythonfile.py
Running python applications through pyspark is not supported as of Spark 2.0.