I am getting the error message in web Activity of ADF:
The request content is not valid and could not be deserialized: 'After parsing a value an unexpected character was encountered:
Further looking into this error, I found that getting an error from notebook which is:
---------------------------------------------------------------------------\nValueError Traceback (most recent call last)\n/tmp/ipykernel_10465/628309876.py in <module>\n 7 \n 8 \n----> 9 split_df.columns = ['dot1', 'dot2', 'dot3', 'dot4', 'dot5']\n 10 print(split_df)\n\n~/cluster-env/env/lib/python3.8/site-packages/pandas/core/generic.py in __setattr__(self, name, value)\n 5476 try:\n 5477 object.__getattribute__(self, name)\n-> 5478 return object.__setattr__(self, name, value)\n 5479 except AttributeError:\n 5480 pass\n\npandas/_libs/properties.pyx in pandas._libs.properties.AxisProperty.__set__()\n\n~/cluster-env/env/lib/python3.8/site-packages/pandas/core/generic.py in _set_axis(self, axis, labels)\n 668 def _set_axis(self, axis: int, labels: Index) -> None:\n 669 labels = ensure_index(labels)\n--> 670 self._mgr.set_axis(axis, labels)\n 671 self._clear_item_cache()\n 672 \n\n~/cluster-env/env/lib/python3.8/site-packages/pandas/core/internals/managers.py in set_axis(self, axis, new_labels)\n 218 \n 219 if new_len != old_len:\n--> 220 raise ValueError(\n 221 f\"Length mismatch: Expected axis has {old_len} elements, new \"\n 222 f\"values have {new_len} elements\"\n\nValueError: Length mismatch: Expected axis has 9 elements, new values have 5 elements"
and when I am sending this error to web activity to send email, I am getting the above web activity error.
Solution Tried:
So, to overcome this error, I created a variable "ErrorMessage" using expression which will hold the error message first:
@activity('Notebook1').error.Message
and then another variable variables('CleanErrorMessage') which will hold clean message replace the \n to '', using
@replace(variables('ErrorMessage'),'\n','')
but it is not working as expected
Can you help me to resolve this and remove this special symbols and line break from the notebook error message so that I can pass the error to web activity in json format and get the result.
Web Activity Json:
{
"Pipeline":"@{pipeline().Pipeline}",
"ErrorMessage": "@{variables('CleanErrorMessage')}",
"DataFactoryName":"@{pipeline().DataFactory}",
"Runid":"@{pipeline().RunId}"
}
You can do the following things to get it worked.
@string(replace(activity('Notebook1').error.Message, '"',''''))