the confusion is because when we specify --sgd in vw command line, it runs classic sgd, without adaptive, normalised and invariant updates. So, when we specify algorithm as sgd in vw-hyperopt, does it run as classic or with special updates? Is it mandatory to specify algorithm in vw-hyperopt? Which is the default algorithm? Thank you.
Looking at the source code confirms that the meaning of --algorithm sgd here simply leaves the default alone.
This is different than vw --sgd. It doesn't disable the defaults by passing --sgd to vw. IOW: yes, the adaptive, normalized and invariant updates will still be in effect.
Also: you can verify this further by looking at the log file created by vw-hyperopt in the current dir and verify it has no --sgd option in it. This log includes the full vw command line it executes for training and testing, e.g:
2020-09-08 00:58:45,053 INFO [root/vw-hyperopt:239]: executing the following command (training): vw -d mydata.train -f ./current.model --holdout_off -c ... --loss_function quantile