Is there a way to interrupt entity update by throwing exception from @PrePersist and @PreUpdate?

I'm looking into JavaDoc of @javax.persistence.PreUpdate and @javax.persistence.PrePersist and there isn't any notice about whether entity persist or update flow will be interrupted when an exception is thrown from the annotated method.

E. g. when I have a listener like

public class EncryptableEntityListener {

    @PrePersist
    @PreUpdate
    private void onUpsert(Object entity) {
        if (!(entity instanceof EncryptableExtended)) {
            return;
        }

        EncryptableExtended encryptableEntity = (EncryptableExtended) entity;

        boolean success = encryptionService.encrypt(encryptableEntity);

        if (!success) {
          throw new RuntimeException():
        }
    }
}

will the transaction be rolled back when success is false?

Yes it will be rolled back. Unless you set it up to not to.