Remove checkout fields for a defined product category in Woocommerce

In woocommerce I am trying to remove unwanted checkout shipping fields for a product category "house".

Here is my code:

 function woo_custom_category_is_in_the_cart( $categories ) {

  // Products currently in the cart
  $cart_ids = array();

  // Categories currently in the cart
  $cart_categories = array('house');

  // Find each product in the cart and add it to the $cart_ids array
  foreach( WC()->cart->get_cart() as $cart_item_key => $values ) {
  $cart_product = $values['data'];
  $cart_ids[]   = $cart_product->id;
  }

  // Connect the products in the cart w/ their categories
  foreach( $cart_ids as $id ) {
  $products_categories = get_the_terms( $id, 'product_cat' );

  // Loop through each product category and add it to our $cart_categories array
  foreach ( $products_categories as $products_category ) {
  $cart_categories[] = $products_category->slug;
  }
  }

  // If one of the special categories are in the cart, return true.
  if ( ! empty( array_intersect( $categories, $cart_categories ) ) ) {
  return true;

  } else {

  return false;

  }

 }
 /************************************************
  * Remove unwanted checkout fields on condition *
  ************************************************/

 function woo_custom_remove_checkout_field( $fields ) {


  $categories  = array( 'house' );

 // If a special category is in the cart, hide the following billing fields
 if ( woo_custom_category_is_in_the_cart( $categories ) ) {


   // hide the billing fields
   unset($fields['shipping']['shipping_first_name']);
  unset($fields['shipping']['shipping_last_name']);
  unset($fields['shipping']['shipping_company']);
  unset($fields['shipping']['shipping_address_1']);
  unset($fields['shipping']['shipping_address_2']);
  unset($fields['shipping']['shipping_city']);
  unset($fields['shipping']['shipping_postcode']);
  unset($fields['shipping']['shipping_country']);
  unset($fields['shipping']['shipping_state']);
  unset($fields['shipping']['shipping_phone']);
  // hide the additional information section
  add_filter('woocommerce_enable_order_notes_field', '__return_false');
  add_filter( 'woocommerce_ship_to_different_address_checked', '__return_false' );
 }
  return $fields;

 }

 add_filter( 'woocommerce_checkout_fields' , 'woo_custom_remove_checkout_field' );

But the code removes the fields on for other product categories too…

What I am doing wrong?

Any help is appreciated.

I have revisited and simplified your code:

add_filter( 'woocommerce_checkout_fields', 'conditionally_remove_checkout_fields', 25, 1 );
function conditionally_remove_checkout_fields( $fields ) {

    // HERE the defined product Categories
    $categories = array('house');

    $found = false;

    // CHECK CART ITEMS: search for items from our defined product category
    foreach ( WC()->cart->get_cart() as $cart_item ){
        if( has_term( $categories, 'product_cat', $cart_item['product_id'] ) ) {
            $found = true;
            break;
        }
    }

    // If a special category is in the cart, remove some shipping fields
    if ( $found ) {

        // hide the billing fields
        unset($fields['shipping']['shipping_first_name']);
        unset($fields['shipping']['shipping_last_name']);
        unset($fields['shipping']['shipping_company']);
        unset($fields['shipping']['shipping_address_1']);
        unset($fields['shipping']['shipping_address_2']);
        unset($fields['shipping']['shipping_city']);
        unset($fields['shipping']['shipping_postcode']);
        unset($fields['shipping']['shipping_country']);
        unset($fields['shipping']['shipping_state']);
        unset($fields['shipping']['shipping_phone']);

        // hide the additional information section
        add_filter('woocommerce_enable_order_notes_field', '__return_false');
        add_filter( 'woocommerce_ship_to_different_address_checked', '__return_false' );
    }
    return $fields;
}

Code goes in function.php file of your active child theme (or active theme). Tested and works.