in TestNG how is the dependonfunction works for the inheritance

Suppose I defined two class BaseClass and DerivedClass below. The waitPeriod for base is 1 minutes, and the waitPeriod for derived is 5 minutes.

class BaseClass {
    protected String getDatasetType() {
        return "BaseClass";
    }

    @BeforeClass
    public void initialize() {
        //some initialize
    }
    @Test
    public void waitPeriod() {
        long a = 1000*300; //wait 5 minutes
        if (getDatasetType().equals("BaseClass")) {
            a = 1000*60; // wait 1 minutes
        }
        try {
            Thread.sleep(a);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        log.info("waitPeriod " + getDatasetType());
    }

    @Test(dependsOnMethods = {"waitPeriod"})
    public void test() {
        log.info("test " + getDatasetType());
    }
}

public class DerivedClass extends BaseClass {
    @Override
    protected String getDatasetType() { return "DerivedClass"; }

}

If run both class in the TestNG, ideally once after 1 minutes, it would only trigger the BaseClass's "test" function, and after 5 minutes, it would trigger the DerivedClass's "test" function. But during my test run, it triggered both of the test function after 1 minute.

As per your code, you have added test() method in BaseClass. So when method is called with @Test annotation, getDatasetType() will be called form BaseClass only.

If you want to achieve expected result, I would suggest you to seperate out tests in other calls, called it as testclass and keep implementation in BaseClass and DerivedClass

Code will look like this

  class BaseClass {
    protected String getDatasetType() {
        return "BaseClass";
    }
    public void initialize() {
        //some initialize
    }
    public void waitPeriod() {
        long a = 1000*300; //wait 5 minutes
        if (getDatasetType().equals("BaseClass")) {
            a = 1000*60; // wait 1 minutes
        }
        try {
            Thread.sleep(a);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        log.info("waitPeriod " + getDatasetType());
     }
  }


  public class DerivedClass extends BaseClass {
      @Override
      protected String getDatasetType() { return "DerivedClass"; }
  }

public class TestClass {
  BaseClass baseClass = new BaseClass();
  DerivedClass derivedClass  = new DerivedClass();

  @BeforeTest
  public void initialize() {
    baseClass.initialize();
  }
  @Test
  public void test_waitPeriod() {
    baseClass.waitPeriod();
  }
  @Test(dependsOnMethods = {"waitPeriod"})
  public void test_baseclass() {
    baseClass.test();
  }
  @Test(dependsOnMethods = {"waitPeriod"})
  public void test_derivedclass() {
    derivedClass.test();
  }
}

// other way can be
public class TestClass {
  BaseClass baseClass = new BaseClass();
  DerivedClass derivedClass  = new DerivedClass();

  @BeforeTest
  public void initialize() {
    baseClass.initialize();
  }

  @Test
  public void test_baseclass() {
    baseClass.waitPeriod();
    baseClass.test();
  }
  @Test
  public void test_derivedclass() {
    derivedClass.waitPeriod();
    derivedClass.test();
  }
}

This is dummy code. You can improve it as per your requirement but try to separate Tests from main implementation.