I'm using tensorflow object detection API, and I want to be able to edit config file dynamically in python, which looks like this. I thought of using protocol buffers library in python, but I'm not sure how to go about.
model {
ssd {
num_classes: 1
image_resizer {
fixed_shape_resizer {
height: 300
width: 300
}
}
feature_extractor {
type: "ssd_inception_v2"
depth_multiplier: 1.0
min_depth: 16
conv_hyperparams {
regularizer {
l2_regularizer {
weight: 3.99999989895e-05
}
}
initializer {
truncated_normal_initializer {
mean: 0.0
stddev: 0.0299999993294
}
}
activation: RELU_6
batch_norm {
decay: 0.999700009823
center: true
scale: true
epsilon: 0.0010000000475
train: true
}
}
...
...
}
Is there a simple/easy way to change specific values for fields like height in image_resizer -> fixed_shape_resizer from say 300 to 500? And write back the file with modified values without changing anything else?
EDIT: Though answer provided by @DmytroPrylipko worked for most of the parameters in the config, I face some issues with "composite field"..
That is, if we have configuration like:
train_input_reader: {
label_map_path: "/tensorflow/data/label_map.pbtxt"
tf_record_input_reader {
input_path: "/tensorflow/models/data/train.record"
}
}
And I add this line to edit input_path:
pipeline_config.train_input_reader.tf_record_input_reader.input_path = "/tensorflow/models/data/train100.record"
It throws error:
TypeError: Can't set composite field
Yes, using Protobuf Python API is quite easy:
edit_pipeline.py:
import argparse
import tensorflow as tf
from google.protobuf import text_format
from object_detection.protos import pipeline_pb2
def parse_arguments():
parser = argparse.ArgumentParser(description='')
parser.add_argument('pipeline')
parser.add_argument('output')
return parser.parse_args()
def main():
args = parse_arguments()
pipeline_config = pipeline_pb2.TrainEvalPipelineConfig()
with tf.gfile.GFile(args.pipeline, "r") as f:
proto_str = f.read()
text_format.Merge(proto_str, pipeline_config)
pipeline_config.model.ssd.image_resizer.fixed_shape_resizer.height = 300
pipeline_config.model.ssd.image_resizer.fixed_shape_resizer.width = 300
config_text = text_format.MessageToString(pipeline_config)
with tf.gfile.Open(args.output, "wb") as f:
f.write(config_text)
if __name__ == '__main__':
main()
The way I call the script:
TOOL_DIR=tool/tf-models/research
(
cd $TOOL_DIR
protoc object_detection/protos/*.proto --python_out=.
)
export PYTHONPATH=$PYTHONPATH:$TOOL_DIR:$TOOL_DIR/slim
python3 edit_pipeline.py pipeline.config pipeline_new.config
Composite fields
In case of repeated fields, you must treat them as arrays (e.g. use extend()
, append()
methods):
pipeline_config.train_input_reader.tf_record_input_reader.input_path[0] = '/tensorflow/models/data/train100.record'
Eval Input reader error
This is a common error trying to edit the composite field. ( "no attribute tf_record_input_reader found" in case of eval_input_reader )
It's mentioned below in @latida's answer. Fix that by setting it as an array field.
pipeline_config.eval_input_reader[0].label_map_path = label_map_full_path
pipeline_config.eval_input_reader[0].tf_record_input_reader.input_path[0] = val_record_path