Creating weights file for new zoo agents about roboschool

[peabody124]

For what it is worth, I'm working on a new environment that creates a simple quadcopter model and allows learning to fly this.

I'm not sure if it would be of interest to contribute.

[olegklimov]

Hi @peabody124. These files exported directly by learning tool. You can tell by the difference in -v0 and -v1 we're working on tools as well. No special script to do this.

[peabody124]

@olegklimov which learning tool, or is this something not released? for example the graphs for flag runner harder say they are using PPO. This algorithm from baselines is working fairly well in my environments, but I'm not exactly sure how to dump the parameters from the MlpPolicy into a script like that which can be executed to reload the weights.

[olegklimov]

Oh, from baselines is really straightforward to export this.

[peabody124]

@olegklimov any hints or links? :)

[olegklimov]

Yeah, sure.

def recursive_np_array_print(a, idx):
    if len(idx)==len(a.shape)-1:
        return "[" + ",".join(["%+8.4f" % a[tuple(idx + [i])] for i in range(a.shape[len(idx)]) ]) + "]"
    else:
        return "[\n" + ",\n".join([recursive_np_array_print(a, idx + [i]) for i in range(a.shape[len(idx)]) ]) + "\n]"

[peabody124]

For what it is worth, this was what I ultimately used to dump from a PPO MlpPolicy to create the weights file

model_meta_fn = 'FILENAME.cpkt.meta'

import tensorflow as tf
sess = tf.Session()
saver = tf.train.import_meta_graph(model_meta_fn)
saver.restore(sess, tf.train.latest_checkpoint('.'))
graph = tf.get_default_graph()

print("Model loaded")

def get_var_by_name(desired):
	return [v for v in tf.global_variables() if v.name == desired][0]


state_mean = get_var_by_name('pi/obfilter/runningsum:0') / get_var_by_name('pi/obfilter/count:0')
state_std = tf.sqrt(tf.maximum(get_var_by_name('pi/obfilter/runningsumsq:0') / get_var_by_name('pi/obfilter/count:0') - tf.square(state_mean), 1e-2))
policy = {  'state_means': sess.run(state_mean),
			'state_std': sess.run(state_std),
			'w0': sess.run(get_var_by_name('pi/polfc1/w:0')),
			'b0': sess.run(get_var_by_name('pi/polfc1/b:0')),
			'w1': sess.run(get_var_by_name('pi/polfc2/w:0')),
			'b1': sess.run(get_var_by_name('pi/polfc2/b:0')),
			'w2': sess.run(get_var_by_name('pi/polfinal/w:0')),
			'b2': sess.run(get_var_by_name('pi/polfinal/b:0'))
		  }

def recursive_np_array_print(a, idx):
	if len(idx)==len(a.shape)-1:
		return "[" + ",".join(["%+8.4f" % a[tuple(idx + [i])] for i in range(a.shape[len(idx)]) ]) + "]"
	else:
		return "[\n" + ",\n".join([recursive_np_array_print(a, idx + [i]) for i in range(a.shape[len(idx)]) ]) + "\n]"

str = "import numpy as np\n"
for key in policy:
	str += key + " = np.array(" + recursive_np_array_print(policy[key], []) + ")\n"

file = open('model.weights', "w")
file.write(str)
file.close()

print("Policy weights dumped to model.weights")

[wenyijiang]

Hi! @peabody124
In the agent_zoo directory there are some files about weights. I want to use my weights that trained by the tensorflow about baselines.ddpg algorithm but there are some issues. I want to know whether I can change my weights file and load it .and how did you do that?
Thanks a lot !!!

[wenyijiang]

@peabody124 I run the code above and change my model file 'model.ckpt.meta' to 'model.weights' . My model is from ddpg algorithm .But there are some issues :
state_mean = get_var_by_name('pi/obfilter/runningsum:0') / get_var_by_name('pi/obfilter/count:0')
return [v for v in tf.global_variables() if v.name == desired][0] IndexError:list index out of range
Could you please tell me the reason and how to modify It ？
Thanks a lot !!

[charles-blouin]

Hi @peabody124 . Would would mind making the quadcopter model available? I would really appreciate! I was digging in the c++ python bullet binding to find a function to apply a pure force or a torque to a robot in roboschool, but I could not find anything. Did I miss something?