microsoft / tdqn Goto Github PK

View Code? Open in Web Editor NEW

21.0 7.0 15.0 279 KB

Template-DQN and DRRN agent implementations

License: MIT License

Python 100.00%

tdqn's People

Stargazers

Watchers

Forkers

maksymdel tzs930 bhaskers-blu-org2 taffywrinkle claudiusgonzo davidlloydwright qiming-zou gargisingh7 marccote standardgalactic ydai94 test-mass-forker-org-1 nicolasag daniels-phd

tdqn's Issues

About PER

Hi, could you please tell what kind of PER is this ? Have you ever tried standard PER to benchmark Jericho ?

tdqn/drrn/memory.py

Lines 29 to 64 in 20908da

 class PrioritizedReplayMemory(object): 

 def __init__(self, capacity=100000, priority_fraction=0.0): 

 self.priority_fraction = priority_fraction 

 self.alpha_capacity = int(capacity * priority_fraction) 

 self.beta_capacity = capacity - self.alpha_capacity 

 self.alpha_memory, self.beta_memory = [], [] 

 self.alpha_position, self.beta_position = 0, 0 

 def push(self, is_prior=False, *args): 

 """Saves a transition.""" 

 if self.priority_fraction == 0.0: 

 is_prior = False 

 if is_prior: 

 if len(self.alpha_memory) < self.alpha_capacity: 

 self.alpha_memory.append(None) 

 self.alpha_memory[self.alpha_position] = Transition(*args) 

 self.alpha_position = (self.alpha_position + 1) % self.alpha_capacity 

 else: 

 if len(self.beta_memory) < self.beta_capacity: 

 self.beta_memory.append(None) 

 self.beta_memory[self.beta_position] = Transition(*args) 

 self.beta_position = (self.beta_position + 1) % self.beta_capacity 

 def sample(self, batch_size): 

 if self.priority_fraction == 0.0: 

 from_beta = min(batch_size, len(self.beta_memory)) 

 res = random.sample(self.beta_memory, from_beta) 

 else: 

 from_alpha = min(int(self.priority_fraction * batch_size), len(self.alpha_memory)) 

 from_beta = min(batch_size - int(self.priority_fraction * batch_size), len(self.beta_memory)) 

 res = random.sample(self.alpha_memory, from_alpha) + random.sample(self.beta_memory, from_beta) 

 random.shuffle(res) 

 return res 

 def __len__(self): 

 return len(self.alpha_memory) + len(self.beta_memory)

Missing Installation instructions and errors with Tensorboard

I followed the installation instructions given on the home page. It did not say I needed tensorflow, but do because of tensorboard.

When grabbing the default tensorflow at the time of this issue (2.4), received the errror:

AttributeError: module 'tensorflow.python.pywrap_tensorflow' has no attribute 'EventsWriter'

Any suggestion on the correct tensorflow to grab or can is it possible to update the install instructions on a 'clean' system with the proper versions?

I also had to install 'sentencepiece' which was not listed and 'torch'

Appreciate it and look forward to trying out tdqn.

Can't adapt it to newer jericho versions. No 2.1.0 version was found

Hi. I'm trying to run drrn/train.py, but can't resolve all the issues.
First of all there is no support for jericho with version 2.1.0, if I'm not mistaken. I could only see versions v1.0, 3.1.1, 3.1.2.
For this I just removed the jericho version check in train.py,
Next there is this api mismatch in env.py. For example it complains that there is no 'load_bindings' defined. I substitutited these calls with methods from FrotzEnv, but I'm getting another error related to multiprocessing from vec_env.py:
assert not _current_process._config.get('daemon'),
AssertionError: daemonic processes are not allowed to have children

microsoft / tdqn Goto Github PK

tdqn's People

Stargazers

Watchers

Forkers

tdqn's Issues

About PER

Missing Installation instructions and errors with Tensorboard

Can't adapt it to newer jericho versions. No 2.1.0 version was found

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent

Jobs

	class PrioritizedReplayMemory(object):
	def __init__(self, capacity=100000, priority_fraction=0.0):
	self.priority_fraction = priority_fraction
	self.alpha_capacity = int(capacity * priority_fraction)
	self.beta_capacity = capacity - self.alpha_capacity
	self.alpha_memory, self.beta_memory = [], []
	self.alpha_position, self.beta_position = 0, 0

	def push(self, is_prior=False, *args):
	"""Saves a transition."""
	if self.priority_fraction == 0.0:
	is_prior = False
	if is_prior:
	if len(self.alpha_memory) < self.alpha_capacity:
	self.alpha_memory.append(None)
	self.alpha_memory[self.alpha_position] = Transition(*args)
	self.alpha_position = (self.alpha_position + 1) % self.alpha_capacity
	else:
	if len(self.beta_memory) < self.beta_capacity:
	self.beta_memory.append(None)
	self.beta_memory[self.beta_position] = Transition(*args)
	self.beta_position = (self.beta_position + 1) % self.beta_capacity

	def sample(self, batch_size):
	if self.priority_fraction == 0.0:
	from_beta = min(batch_size, len(self.beta_memory))
	res = random.sample(self.beta_memory, from_beta)
	else:
	from_alpha = min(int(self.priority_fraction * batch_size), len(self.alpha_memory))
	from_beta = min(batch_size - int(self.priority_fraction * batch_size), len(self.beta_memory))
	res = random.sample(self.alpha_memory, from_alpha) + random.sample(self.beta_memory, from_beta)
	random.shuffle(res)
	return res

	def __len__(self):
	return len(self.alpha_memory) + len(self.beta_memory)