Spring boot @ConditionalOnBean does not detect AutoConfiguration bean

I want to ensure that the MyKafkaProducer bean is only activated when KafkaAutoConfiguration is loaded, so I have created the code as follows:

@Service
@ConditionalOnBean(KafkaAutoConfiguration.class)
@RequiredArgsConstructor
public class MyKafkaProducer {
    private final KafkaTemplate<String, KafkaEntity> kafkaTemplate;
    // omitted...
}

I checked the list of beans created after running the code at http://localhost:8080/actuator/beans. KafkaAutoConfiguration was indeed created like below.
But MyKafkaProducer However, was not created. So weird.

        "org.springframework.boot.autoconfigure.kafka.KafkaAutoConfiguration": {
          "aliases": [],
          "scope": "singleton",
          "type": "org.springframework.boot.autoconfigure.kafka.KafkaAutoConfiguration",
          "dependencies": [
            "spring.kafka-org.springframework.boot.autoconfigure.kafka.KafkaProperties"
          ]
        },

I changed the @ConditionalOnBean to @ConditionalOnMissingBean in the code and reran it. This time, both KafkaAutoConfiguration and kafkaDlrObserver bean was successfully created.

        "org.springframework.boot.autoconfigure.kafka.KafkaAutoConfiguration": {
          "aliases": [],
          "scope": "singleton",
          "type": "org.springframework.boot.autoconfigure.kafka.KafkaAutoConfiguration",
          "dependencies": [
            "spring.kafka-org.springframework.boot.autoconfigure.kafka.KafkaProperties"
          ]
        },

        "kafkaDlrObserver": {
          "aliases": [],
          "scope": "singleton",
          "type": "com.my.project.kafka.MyKafkaProducer",
          "resource": "file [/target/classes/com/my/project/kafka/MyKafkaProducer.class]",
          "dependencies": [
            "kafkaProducer",
            "resultMessageFactory"
          ]
        },

Question:

• This means that KafkaAutoConfiguration is not created by default, which is strange because it is showen in the bean list. Why is that?

From the documentation of the ConditionalOnBean class:

The condition can only match the bean definitions that have been processed by the application context so far and, as such, it is strongly recommended to use this condition on auto-configuration classes only.

Putting an @Conditional... annotation on a @Service or other component class as shown in your example is not reliable.

Instead, you should create the bean with an @Bean method on a @AutoConfiguration class, something like this:

@RequiredArgsConstructor
public class MyKafkaProducer {
    private final KafkaTemplate<String, KafkaEntity> kafkaTemplate;
    // omitted...
}

@AutoConfiguration
@ConditionalOnBean(KafkaAutoConfiguration.class)
public class MyKafkaConfiguration {
    @Bean MyKafkaProducer myKafkaProducer(...) {
        return new MyKafkaProducer(...);
    }
}  

See the Spring Boot documentation for more information on creating auto-configuration classes.