I'm doing some machine learning and I have a Generator class that generates text and can do so with several models. I use a Pydantic Model to validate its parameters and give me a type-hinted object that's easy to work with throughout the code.
The problem is that I'd like to be able to have an add_params method in the Generator class that is type hinted just like the pydantic model. But to do this I have to write down the type hints manually in the add_params method. This seems like a recipe for disaster since it's not unlikely that I'll change the type hints and docs for the pydantic model and forget to change the add_params method or visa versa.
Right now, I'm just using kwargs for add_params and referring to the pydantic model when I need to remember what I can pass. But I'm hoping there's a better way to do it.
class GenerationParameters(pydantic.BaseModel):
# list of all possible parameters and types
# and some validation methods
class Generator:
param_list:List[GenerationParameters] = []
def add_params(model_name:str, **kwargs):
# load default params for that model
params_dict = get_default_params_from_model_name(model_name:str)
# overwrite defaults with passed kwargs
params_dict.update(kwargs)
# pass to pydantic model for validation and easy access
pydantic_params = GenerationParameters(model_name=model_name, **params_dict)
# add to params list
self.params_list.append(pydantic_params )
def generate():
output = []
for params in param_list:
output.append(self._generate_from_params(params))
return output
Is there a way to use the type hints for GenerationParameters for add_params kwargs? Or is there a better way to combine these type hints and docs?
Don't make Generator.add_params responsible for constructing an instance of GenerationParameters. Let the caller construct it and pass it to the method, which now simply needs to store it for later use.
class GenerationParameters(pydantic.BaseModel):
# list of all possible parameters and types
# and some validation methods
class Generator:
param_list:List[GenerationParameters] = []
def add_params(gp: GenerationParameters):
self.params_list.append(gp)
def generate(self):
output = []
for params in param_list:
output.append(self._generate_from_params(params))
return output
g = Generator()
g.add_params(GenerationParameters('model1', foo=3, bar=6))
get_default_params_from_model_name should be used by a method of GenerationParameters to help construct the instance.
Now Generator has no responsibility for ensuring GenerationParameters is correctly instantiated.
Using a ** parameter is fundamentally at odds with the idea of static type hinting. You are saying "I don't know, statically, what arguments will be provided. I'll just accepted whatever I am given at run time." The fact that you will pass those unknown arguments to a function that is statically hinted doesn't matter.