kubernetes container_name got null in fluentdconfiguration

I try to get log from my application container and attach fluentd log agent as sidecar container in my project. And I want to get which log is coming from which application in my Kibana dashboard. That's why I configured like that in fluentd.

<source>
  @id fluentd-containers.log
  @type tail
  path /var/log/containers/mylog*.log
  pos_file /var/log/es-containers.log.pos
  time_format %Y-%m-%dT%H:%M:%S.%NZ
  tag kubernetes.myapp.container
  read_from_head true
  <parse>
    @type none
  </parse>
</source>

<filter kubernetes**>
  @type record_transformer
  enable_ruby true
  <record>
    service_name ${tag_parts[1]}
    instance_name ${record["kubernetes"]["container_name"]}
    log_type ${tag_parts[2]}
    host_name ${hostname}
    send_to "ES"
  </record>
</filter>

<match kubernetes.**>
  @type stdout
</match>

But when I deployed it, ${[record[""]["container_name"]} got null as displaying unknown placeholder ${record["kubernetes"]["container_name"]}. Please help me how to resolve it, thanks.

Got that error message

0 dump an error event: error_class=RuntimeError error="failed to expand record[\"kubernetes\"][\"container_name\"] : error = undefined method []' for nil:NilClass" location="/fluentd/vendor/bundle/ruby/2.6.0/gems/fluentd-1.11.2/lib/fluent/plugin/filter_record_transformer.rb:310:in rescue in expand'" tag="kubernetes.myapp.container" time=2020-09-23 11:29:05.705209241 +0000 record={"message"=>"{"log":"I0923 11:28:59.157177 1 main.go:71] Health check succeeded\n","stream":"stderr","time":"2020-09-23T11:28:59.157256887Z"}"}

`

The record doesn't contain the required fields that you want to access i.e. record["kubernetes"]["container_name"].

You need to make sure that it has those fields.

Please go through Container Deployment and kubernetes_metadata_filter plugin for the detailed information on this.