streaminfluxdbkapacitor

Unable to get response http Post to local express app from Kapacitor stream

I am following SE Thread to get some response to HTTP POST on an express node. But unable to get any response from kapacitor.

Environment

I am using Windows 10 via PowerShell. I am connected to an InfluxDB internal Server which is mentioned in the kapacitor.conf and have a TICKscript to stream data via it.

kapacitor.conf

hostname = "134.102.97.81"
data_dir = "C:\\Users\\des\\.kapacitor"
skip-config-overrides = true
default-retention-policy = ""

[alert]
  persist-topics = true

[http]
  bind-address = ":9092"
  auth-enabled = false
  log-enabled = true
  write-tracing = false
  pprof-enabled = false
  https-enabled = false
  https-certificate = "/etc/ssl/kapacitor.pem"
  https-private-key = ""
  shutdown-timeout = "10s"
  shared-secret = ""

[replay]
  dir = "C:\\Users\\des\\.kapacitor\\replay"

[storage]
  boltdb = "C:\\Users\\des\\.kapacitor\\kapacitor.db"

[task]
  dir = "C:\\Users\\des\\.kapacitor\\tasks"
  snapshot-interval = "1m0s"

[load]
  enabled = false
  dir = "C:\\Users\\des\\.kapacitor\\load"

[[influxdb]]
  enabled = true
  name = "DB5Server"
  default = true
  urls = ["https://influxdb.internal.server.address:8086"]
  username = "user"
  password = "password"
  ssl-ca = ""
  ssl-cert = ""
  ssl-key = ""
  insecure-skip-verify = true
  timeout = "0s"
  disable-subscriptions = true
  subscription-protocol = "https"
  subscription-mode = "cluster"
  kapacitor-hostname = ""
  http-port = 0
  udp-bind = ""
  udp-buffer = 1000
  udp-read-buffer = 0
  startup-timeout = "5m0s"
  subscriptions-sync-interval = "1m0s"
  [influxdb.excluded-subscriptions]
    _kapacitor = ["autogen"]

[logging]
  file = "STDERR"
  level = "DEBUG"

[config-override]
  enabled = true

[[httppost]]
  endpoint = "kapacitor"
  url = "http://localhost:1440"
  headers = { Content-Type = "application/json;charset=UTF-8"}
  alert-template = "{\"id\": {{.ID}}}"

The daemon runs without any problems.

test2.tick

dbrp "DBTEST"."autogen"

stream
    |from()
        .measurement('humid')
    |alert()
        .info(lambda: TRUE)
        .post()
        .endpoint('kapacitor')
  1. Already defined the task .\kapacitor.exe define bc_1 -tick test2.tick
  2. Enabled it .\kapacitor.exe enable bc_1

The status shows nothing:

.\kapacitor.exe show bc_1



ID: bc_1
Error:
Template:
Type: stream
Status: enabled
Executing: true
Created: 13 Mar 19 15:33 CET
Modified: 13 Mar 19 16:23 CET
LastEnabled: 13 Mar 19 16:23 CET
Databases Retention Policies: ["NIMBLE"."autogen"]
TICKscript:
dbrp "TESTDB"."autogen"

stream
    |from()
        .measurement('humid')
    |alert()
        .info(lambda: TRUE)
        .post()
        .endpoint('kapacitor')

DOT:
digraph bc_1 {
graph [throughput="0.00 points/s"];

stream0 [avg_exec_time_ns="0s" errors="0" working_cardinality="0" ];
stream0 -> from1 [processed="0"];

from1 [avg_exec_time_ns="0s" errors="0" working_cardinality="0" ];
from1 -> alert2 [processed="0"];

alert2 [alerts_inhibited="0" alerts_triggered="0" avg_exec_time_ns="0s" crits_triggered="0" errors="0" infos_triggered="0" oks_triggered="0" warns_triggered="0" working_cardinality="0" ];
}

The Daemon logs provide this for the task

ts=2019-03-13T16:25:23.640+01:00 lvl=debug msg="starting enabled task on startup" service=task_store task=bc_1
ts=2019-03-13T16:25:23.677+01:00 lvl=debug msg="starting task" service=kapacitor task_master=main task=bc_1
ts=2019-03-13T16:25:23.678+01:00 lvl=info msg="started task" service=kapacitor task_master=main task=bc_1
ts=2019-03-13T16:25:23.679+01:00 lvl=debug msg="listing dot" service=kapacitor task_master=main dot="digraph bc_1 {\nstream0 -> from1;\nfrom1 -> alert2;\n}"
ts=2019-03-13T16:25:23.679+01:00 lvl=debug msg="started task during startup" service=task_store task=bc_1
ts=2019-03-13T16:25:23.680+01:00 lvl=debug msg="opened service" source=srv service=*task_store.Service
ts=2019-03-13T16:25:23.680+01:00 lvl=debug msg="opening service" source=srv service=*replay.Service
ts=2019-03-13T16:25:23.681+01:00 lvl=debug msg="skipping recording, metadata is already correct" service=replay recording_id=353d8417-285d-4fd9-b32f-15a82600f804
ts=2019-03-13T16:25:23.682+01:00 lvl=debug msg="skipping recording, metadata is already correct" service=replay recording_id=a8bb5c69-9f20-4f4d-8f84-109170b6f583

But I get nothing on the Express Node side. The code is exactly the same as that in the above mentioned SE thread.

Any Help as to how to capture stream from Kapacitor on HTTP Post? I already have a live system that is pushing information into the dedicated database already

Solution

  • I was able to shift focus from stream to batch in the above query. I have documented the complete process on medium.com.

    Some Files:

    kapacitor.gen.conf

    hostname = "my-windows-10"
data_dir = "C:\\Users\\<user>\\.kapacitor"
skip-config-overrides = true
default-retention-policy = ""

[alert]
  persist-topics = true

[http]
  bind-address = ":9092"
  auth-enabled = false
  log-enabled = true
  write-tracing = false
  pprof-enabled = false
  https-enabled = false
  https-certificate = "/etc/ssl/kapacitor.pem"
  https-private-key = ""
  shutdown-timeout = "10s"
  shared-secret = ""

[replay]
  dir = "C:\\Users\\des\\.kapacitor\\replay"

[storage]
  boltdb = "C:\\Users\\des\\.kapacitor\\kapacitor.db"

[task]
  dir = "C:\\Users\\des\\.kapacitor\\tasks"
  snapshot-interval = "1m0s"

[load]
  enabled = false
  dir = "C:\\Users\\des\\.kapacitor\\load"

[[influxdb]]
  enabled = true
  name = "default"
  default = true
  urls = ["http://127.0.0.1:8086"]
  username = ""
  password = ""
  ssl-ca = ""
  ssl-cert = ""
  ssl-key = ""
  insecure-skip-verify = true
  timeout = "0s"
  disable-subscriptions = true
  subscription-protocol = "http"
  subscription-mode = "cluster"
  kapacitor-hostname = ""
  http-port = 0
  udp-bind = ""
  udp-buffer = 1000
  udp-read-buffer = 0
  startup-timeout = "5m0s"
  subscriptions-sync-interval = "1m0s"
  [influxdb.excluded-subscriptions]
    _kapacitor = ["autogen"]

[logging]
  file = "STDERR"
  level = "DEBUG"

[config-override]
  enabled = true

# Subsequent Section describes what this conf does

[[httppost]]
  endpoint = "kap"
  url = "http://127.0.0.1:30001/kapacitor"
  headers = { "Content-Type" = "application/json"}

    TICKScript

    var data =  batch
     | query('SELECT "v" FROM "telegraf_test"."autogen"."humid"')
       .period(5s)
       .every(10s)

data
      |httpPost()
       .endpoint('kap')

    Define the Task

    .\kapacitor.exe define batch_test -tick .\batch_test.tick -dbrp DBTEST.autogen

    I suspect the hostname was michieveous where it was set to localhost previously but I set it my machine's hostname and instead used the IP address 127.0.0.1 whereever localhost was mentioned