pythontensorflowrllib

Printing model summaries for rllib models


I have not seen anything in the rllib documentation that would allow me to print a quick summary of the model like print(model.summary()) in keras. I tried using tf-slim and

variables = tf.compat.v1.model_variables()
slim.model_analyzer.analyze_vars(variables, print_info=True)

to get a rough idea for tensorflow models, but this found no variables after the model was initialized (inserted at the end of the ESTrainer class _init). Specifically, I have been trying to get a summary of an Evolutionary Strategy (ES) policy to verify that the changes to the model config are being updated as expected, but I have not been able to get a summary print working.

Is there an existing method for this? Is slim expected to work here?


Solution

  • The training agent can return the policy which gives you access to the model:

    agent = ppo.PPOTrainer(config, env=select_env)
    
    policy = agent.get_policy()
    policy.model.base_model.summary() # Prints the model summary
    

    Sample output:

     Layer (type)                   Output Shape         Param #     Connected to                     
    ==================================================================================================
     observations (InputLayer)      [(None, 7)]          0           []                               
                                                                                                      
     fc_1 (Dense)                   (None, 256)          2048        ['observations[0][0]']           
                                                                                                      
     fc_value_1 (Dense)             (None, 256)          2048        ['observations[0][0]']           
                                                                                                      
     fc_2 (Dense)                   (None, 256)          65792       ['fc_1[0][0]']                   
                                                                                                      
     fc_value_2 (Dense)             (None, 256)          65792       ['fc_value_1[0][0]']             
                                                                                                      
     fc_out (Dense)                 (None, 5)            1285        ['fc_2[0][0]']                   
                                                                                                      
     value_out (Dense)              (None, 1)            257         ['fc_value_2[0][0]']             
                                                                                                      
    ==================================================================================================
    Total params: 137,222
    Trainable params: 137,222
    Non-trainable params: 0