javascriptjquerystreamsets

Streamsets Javascript scripting error loading JQuery


I am trying to load JQuery in the Streamsets Javascript Scripting Origin to query rest api and getting an error. The code looks like below and java load permission was given in /etc/sdc/sdc-security.policy. Anyone have sample Streamsets javascript code to query rest api?

load('file:///opt/streamsets-datacollector/resources/jquery-3.5.1.min.js');

var settings = {
  "url": "https://example.com/abc?parm1=123&parm2=456",
  "method": "GET",
  "timeout": 0,
  "headers": {
    "Accept": "application/vnd.mds.provider+json;version=0.3",
    "Authorization": "Bearer some-token-xxxxx"
  },
};

$.ajax(settings).done(function (response) {
  console.log(response);
});

Below is the stack error:

com.streamsets.pipeline.api.StageException: SCRIPTING_10 - Script error in user script: javax.script.ScriptException: TypeError: Cannot read property "createElement" from undefined in file:/opt/streamsets-datacollector/resources/jquery-3.5.1.min.js at line number 2
    at com.streamsets.pipeline.stage.origin.scripting.AbstractScriptingSource.produce(AbstractScriptingSource.java:137)
    at com.streamsets.pipeline.api.base.configurablestage.DPushSource.produce(DPushSource.java:44)
    at com.streamsets.datacollector.runner.StageRuntime.lambda$execute$1(StageRuntime.java:270)
    at com.streamsets.datacollector.runner.StageRuntime.execute(StageRuntime.java:244)
    at com.streamsets.datacollector.runner.StageRuntime.execute(StageRuntime.java:279)
    at com.streamsets.datacollector.runner.SourcePipe.process(SourcePipe.java:79)
    at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.runPushSource(ProductionPipelineRunner.java:426)
    at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.run(ProductionPipelineRunner.java:388)
    at com.streamsets.datacollector.runner.Pipeline.run(Pipeline.java:533)



Solution

  • JQuery is usually used for HTML document traversal. You should look into using HTTP Client origin instead to invoke the REST API endpoint.

    Cheers, Dash