Error while try getting spark connection id in Airflow
My SparkSubmitOperator in airflow DAG looks as below, below is my connection_id 'spark_local' in Airflow UI as 
when I try running my DAG am getting this error 
, can anyone please help me if am missing anything here?
SparkSubmitOperator (task_id='spark_task', application = 'gs://xxx/xxx.jar', conf = {"spark.driver.allowMultipleContexts":True, "spark.blacklist.enabled":False}, conn_id='spark_local', java_class = 'xxx', jars=["gs://xxx/*"], application_args=["xxx "xxx" "xxx,] )
Based on the TypeError , from the documentation https://airflow.apache.org/docs/apache-airflow/1.10.12/_api/airflow/contrib/operators/spark_submit_operator/index.html, it said that jars argument should be a string. Could you try jars="gs://xxx/*" as a string instead to see if it's working?
- What is the best way to get the last non zero value in a window of N rows?
- GET http://localhost:8000/project:slug
- Why is the bounding box not aligned to the square?
- Why is my manim scene being converted into a png instead of an mp4
- Can't install numpy with PyPy 7.3.19
- How to convert rows to columns in a Pandas groupby?
- UUID('...') is not JSON serializable
- How to have a Zoom API that shows my history
- New file on sharepoint, trigger python script
- LLDB breaks on adding a breakpoint using python
- Using Python Pyadomd Library to get PowerBI Data into Json Format
- Filter Pandas dataframe based on combination of two columns
- How to avoid "object has no attribute 'isAlive'" error while debugging in IntelliJ using Python plugin version 241.18034.62
- Using Ruff Linter with Python in VS Code
- AttributeError: 'list' object has no attribute 'encode'. How do i send Email via Excel? Python
- How can I integrate Python mido and asyncio?
- How do I prevent fixtures from conflicting with django post_save signal code?
- Python randomly generated IP address as string
- How to find the importance of the features for a logistic regression model?
- How to run a Python 3 tkinter app without opening any Terminal windows in Mac OS X?
- Remove spurious small islands of noise in an image - Python OpenCV
- Gamma correction with Pillow Image turning to Grayscale
- Min Max Image Histogram Stretching
- Linear Regression on each column without creating for loops or functions
- What is the purpose of the "send" function on Python generators?
- Python GPS Module: Reading latest GPS Data
- Can I pass command-line arguments to Abaqus python?
- How to make a list of JSON objects into a JSON file in Python and join multiple ones?
- Python on the Web
- Program for Python that traces back code of a function call or instantiation and nicely prints it all out