danieltakeshi / paper_notes

This will contain my notes for research papers that I read.

Inspired by Adrian Colyer and Denny Britz.

This contains my notes for research papers that I've read. Papers are arranged according to three broad categories and then further numbered on a (1) to (5) scale where a (1) means I have only barely skimmed it, while a (5) means I feel confident that I understand almost everything about the paper. Within a single year, these papers should be organized according to publication date. The links here go to my paper summaries if I have them, otherwise those papers are on my TODO list.

Contents:

• Reinforcement and Imitation Learning
  • 2019 Papers
  • 2018 Papers
  • 2017 Papers
  • 2016 Papers
  • 2015 Papers
  • 2014 and Earlier
• Deep Learning
  • 2019 Papers
  • 2018 Papers
  • 2017 Papers
  • 2016 Papers
  • 2015 Papers
  • 2014 and Earlier
• Miscellaneous
  • 2019 Papers
  • 2018 Papers
  • 2017 Papers
  • 2016 Papers
  • 2015 Papers
  • 2014 Papers
  • 2013 and Earlier

Reinforcement Learning and Imitation Learning

2019 RL/IL Papers

• Extending Deep MPC with Safety Augmented Value Estimation from Demonstrations, arXiv 2019 (3)
• Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reduction, arXiv 2019 (1)
• SQIL: Imitation Learning via Regularized Behavioral Cloning, arXiv 2019 (1)
• Towards Characterizing Divergence in Deep Q-Learning, arXiv 2019 (1)
• Skew-Fit: State-Covering Self-Supervised Reinforcement Learning, arXiv 2019 (1)
• Visual Hindsight Experience Replay, arXiv 2019 (1)
• Diagnosing Bottlenecks in Deep Q-Learning Algorithms, ICML 2019 (1)
• Efficient Off-Policy Meta-Reinforcement learning via Probabilistic Context Variables, ICML 2019 (1)
• Off-Policy Deep Reinforcement Learning Without Exploration ICML 2019 (5)

Early-year

• Model-Based Reinforcement Learning for Atari, arXiv 2019 (1)
• Reinforcement Learning from Imperfect Demonstrations, arXiv 2019 (1)
• Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control, arXiv (4)
• Residual Reinforcement Learning for Robot Control, ICRA 2019 (3)
• Memory Efficient Experience Replay for Streaming Learning, ICRA 2019 (1)
• Learning Curriculum Policies for Reinforcement Learning, AAMAS 2019 (1)
• Variational Discriminator Bottleneck: Improving IL, IRL, and GANs by Constraining Information Flow, ICLR 2019 (1)
• Recurrent Experience Replay in Distributed Reinforcement Learning, ICLR 2019 (1)
• InfoBot: Transfer and Exploration via the Information Bottleneck, ICLR 2019 (1)
• Large-Scale Study of Curiosity-Driven Learning, ICLR 2019 (4)
• Exploration by Random Network Distillation, ICLR 2019 (4)
• Automatically Composing Representation Transformations as a Means for Generalization, ICLR 2019 (2)
• Diversity is All You Need: Learning Skills without a Reward Function, ICLR 2019 (4)
• Distilling Policy Distillation, AISTATS 2019 (1)
• Multi-task Deep Reinforcement Learning with PopArt, AAAI 2019 (4)

2018 RL/IL Papers

Late-year

• Simple Random Search Provides a Competitive Approach to Reinforcement Learning, NeurIPS 2018 (5)
• Learning to Play with Intrinsically-Motivated Self-Aware Agents NeurIPS 2018 (2)
• Reward Learning from Human Preferences and Demonstrations in Atari, NeurIPS 2018 (3)
• Improving Exploration in ES for DeepRL via a Population of Novelty-Seeking Agents, NeurIPS 2018 (3)
• Visual Reinforcement Learning with Imagined Goals, NeurIPS 2018 (1)
• Probabilistic Model-Agnostic Meta-Learning, NeurIPS 2018 (1)
• Playing Hard Exploration Games by Watching YouTube, NeurIPS 2018 (1)
• Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models, NeurIPS 2018 (1)
• Sim-to-Real Reinforcement Learning for Deformable Object Manipulation, CoRL 2018 (4)
• GPU-Accelerated Robotic Simulation for Distributed Reinforcement Learning, CoRL 2018 (1)
• Learning Actionable Representations from Visual Observations, IROS 2018 (4)
• Natural Environment Benchmarks for Reinforcement Learning, arXiv 2018 (1)

Mid-year

• Learning Instance Segmentation by Interaction, CVPR 2018 (1)
• Learning by Playing –- Solving Sparse Reward Tasks from Scratch, ICML 2018 (1)
• IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures, ICML 2018 (2)
• Universal Planning Networks, ICML 2018 (1)
• Hierarchical Imitation and Reinforcement Learning, ICML 2018 (1)
• Progress & Compress: A Scalable Framework for Continual Learning, ICML 2018 (1)
• Policy Optimization with Demonstrations, ICML 2018 (1)
• Investigating Human Priors for Playing Video Games, ICML 2018 (3)
• Self-Imitation Learning, ICML 2018 (4)
• Automatic Goal Generation for Reinforcement Learning Agents, ICML 2018 (4)
• Reinforcement and Imitation Learning for Diverse Visuomotor Skills, RSS 2018 (2)
• One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learning, RSS 2018 (4)
• Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations, RSS 2018 (4)
• Asymmetric Actor Critic for Image-Base Robot Learning, RSS 2018 (5)
• Differentiable Physics and Stable Modes for Tool-Use and Manipulation Planning, RSS 2018 (1)
• Kickstarting Deep Reinforcement Learning, arXiv 2018 (5)
• Observe and Look Further: Achieving Consistent Performance on Atari, arXiv 2018 (4)

Early-year

• Time-Contrastive Networks: Self-Supervised Learning from Video, ICRA 2018 (1)
• Neural Network Dynamics for Model-Based Deep RL with Model-Free Fine-Tuning, ICRA 2018 (5)
• Learning Robotic Assembly from CAD, ICRA 2018 (3)
• CASSL: Curriculum Accelerated Self-Supervised Learning, ICRA 2018 (5)
• Neural Task Programming: Learning to Generalize Across Hierarchical Tasks, ICRA 2018 (2)
• Overcoming Exploration in Reinforcement Learning with Demonstrations, ICRA 2018 (5)
• Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation, ICRA 2018 (3)
• Parameterized Hierarchical Procedures for Neural Programming, ICLR 2018 (4)
• Meta Learning Shared Hierarchies, ICLR 2018 (5)
• Divide-and-Conquer Reinforcement Learning, ICLR 2018 (3)
• Zero-Shot Visual Imitation, ICLR 2018 (4)
• Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Play, ICLR 2018 (3)
• Emergent Complexity via Multi-Agent Competition, ICLR 2018 (3)
• Progressive Reinforcement Learning With Distillation For Multi-Skilled Motion Control, ICLR 2018 (4)
• Model-Ensemble Trust-Region Policy Optimization, ICLR 2018 (3)
• Distributed Prioritized Experience Replay, ICLR 2018 (4)
• Learning to Multi-Task by Active Sampling, ICLR 2018 (1)
• Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learning, ICLR 2018 (1)
• Rainbow: Combining Improvements in Deep Reinforcement Learning, AAAI 2018 (4)
• Deep Q-learning from Demonstrations, AAAI 2018 (5)
• Deep Reinforcement Learning that Matters, AAAI 2018 (4)

2017 RL/IL Papers

Late-year

• Deep Reinforcement Learning from Human Preferences, NeurIPS 2017 (3)
• One-Shot Imitation Learning, NeurIPS 2017 (4)
• #Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learning, NeurIPS 2017 (4)
• Robust Imitation of Diverse Behaviors, NeurIPS 2017 (3)
• Bridging the Gap Between Value and Policy Based Reinforcement Learning, NeurIPS 2017 (2)
• Inferring The Latent Structure of Human Decision-Making from Raw Visual Inputs, NeurIPS 2017 (5)
• Hindsight Experience Replay, NeurIPS 2017 (5)
• Distral: Robust Multitask Reinforcement Learning, NeurIPS 2017 (1)
• DART: Noise Injection for Robust Imitation Learning, CoRL 2017 (3)
• Learning Deep Policies for Robot Bin Picking by Simulating Robust Grasping Sequences, CoRL, 2017 (3)
• DDCO: Discovery of Deep Continuous Options for Robot Learning from Demonstrations, CoRL 2017 (5)
• Reverse Curriculum Generation for Reinforcement Learning, CoRL 2017 (5)
• One-Shot Visual Imitation Learning via Meta-Learning, CoRL 2017 (5)
• Sim-to-Real Robot Learning from Pixels with Progressive Nets, CoRL 2017 (1)
• Transferring End-to-End Visuomotor Control from Simulation to Real World for a Multi-Stage Task, CoRL 2017 (1)
• Visual Semantic Planning using Deep Successor Representations, ICCV 2017 (4)
• Proximal Policy Optimization Algorithms, arXiv (4)
• Learning Human Behaviors From Motion Capture by Adversarial Imitation, arXiv (3)
• The Uncertainty Bellman Equation and Exploration, arXiv (1)
• Multi-Level Discovery of Deep Options, arXiv (5)
• Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards, arXiv (5)

Mid-year

• Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World, IROS 2017 (4)
• Virtual to Real Reinforcement Learning for Autonomous Driving, BMVC 2017 (3)
• ReasoNet: Learning to Stop Reading in Machine Comprehension, KDD 2017 (3)
• Inverse Reinforcement Learning via Deep Gaussian Process, UAI 2017 (2)
• Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learning, ICML 2017 (1)
• Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observability, ICML 2017 (1)
• Reinforcement Learning with Deep Energy-Based Policies, ICML 2017 (1)
• A Distributional Perspective on Reinforcement Learning, ICML 2017 (2)
• Robust Adversarial Reinforcement Learning, ICML 2017 (5)
• Modular Multitask Reinforcement Learning with Policy Sketches, ICML 2017 (4)
• End-to-End Differentiable Adversarial Imitation Learning, ICML 2017 (4)
• Constrained Policy Optimization, ICML 2017 (2)
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks, ICML 2017 (4)
• Curiosity-Driven Exploration by Self-Supervised Prediction, ICML 2017 (4)
• Towards End-to-End Reinforcement Learning of Dialogue Agents for Information Access, ACL 2017 (3)
• Unsupervised Perceptual Rewards for Imitation Learning, RSS 2017 (1)
• Loss is its own Reward: Self-Supervision for Reinforcement Learning, arXiv (2)
• Evolution Strategies as a Scalable Alternative to Reinforcement Learning, arXiv (5)

Early-year

• Imitating Driver Behavior with Generative Adversarial Networks, Intelligent Vehicles (IV), 2017 (4)
• Reinforcement Learning with Unsupervised Auxiliary Tasks, ICLR 2017 (3)
• Learning to Repeat: Fine-Grained Action Repetition for Deep Reinforcement Learning, ICLR 2017 (4)
• Learning to Act by Predicting the Future, ICLR 2017 (4)
• Learning Visual Servoing with Deep Features and Fitted Q-Iteration, ICLR 2017 (2)
• Q-Prop: Sample-Efficient Policy Gradient with an Off-Policy Critic, ICLR 2017 (2)
• Stochastic Neural Networks for Hierarchical Reinforcement Learning, ICLR 2017 (4)
• Generalizing Skills With Semi-Supervised Reinforcement Learning ICLR 2017 (3)
• Third-Person Imitation Learning, ICLR 2017 (3)
• Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates ICRA 2017 (5)
• Target-Driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning, ICRA 2017 (5)
• Supervision via Competition: Robot Adversaries for Learning Tasks, ICRA 2017 (4)
• Deep Visual Foresight for Planning Robot Motion, ICRA 2017 (3)
• Comparing Human-Centric and Robot-Centric Sampling for Robot Deep Learning from Demos, ICRA 2017 (4)
• Combining Self-Supervised Learning and Imitation for Vision-Based Rope Manipulation, ICRA 2017 (5)
• Learning to Push by Grasping: Using Multiple Tasks for Effective Learning, ICRA 2017 (4)
• Learning Modular Neural Network Policies for Multi-Task and Multi-Robot Transfer, ICRA 2017 (4)
• Dynamic Action Repetition for Deep Reinforcement Learning, AAAI 2017 (5)
• Knowledge Transfer for Deep Reinforcement Learning with Hierarchical Experience Replay, AAAI 2017 (4)
• A Deep Hierarchical Approach to Lifelong Learning in Minecraft, AAAI 2017 (4)
• RL^2: Fast Reinforcement Learning via Slow Reinforcement Learning, arXiv (4)
• Learning to Predict Where to Look in Interactive Environments Using Deep Recurrent Q-Learning, arXiv (3)

2016 RL/IL Papers

• Learning to Poke by Poking: Experiential Learning of Intuitive Physics, NeurIPS 2016 (5)
• Value Iteration Networks, NeurIPS 2016 (4)
• Generative Adversarial Imitation Learning, NeurIPS 2016 (3)
• VIME: Variational Information Maximizing Exploration, NeurIPS 2016 (3)
• Unsupervised Learning for Physical Interaction through Video Prediction, NeurIPS 2016 (1)
• Deep Exploration via Bootstrapped DQN, NeurIPS 2016 (3)
• Unifying Count-Based Exploration and Intrinsic Motivation, NeurIPS 2016 (1)
• Principled Option Learning in Markov Decision Processes, EWRL 2016 (4)
• Taming the Noise in Reinforcement Learning via Soft Updates, UAI 2016 (4)
• Deep Successor Reinforcement Learning, arXiv 2016 (4)
• Terrain-Adaptive Locomotion Skills Using Deep Reinforcement Learning, SIGGRAPH 2016 (3)
• Asynchronous Methods for Deep Reinforcement Learning, ICML 2016 (4)
• Benchmarking Deep Reinforcement Learning for Continuous Control, ICML 2016 (4)
• Model-Free Imitation Learning with Policy Optimization, ICML 2016 (4)
• Graying the Black Box: Understanding DQNs, ICML 2016 (4)
• Control of Memory, Active Perception, and Action in Minecraft, ICML 2016 (2)
• Dueling Network Architectures for Deep Reinforcement Learning, ICML 2016 (4)
• Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization, ICML 2016 (2)
• Policy Distillation, ICLR 2016 (5)
• Learning Visual Predictive Models of Physics for Playing Billiards, ICLR 2016 (4)
• Prioritized Experience Replay, ICLR 2016 (4)
• High-Dimensional Continuous Control Using Generalized Advantage Estimation, ICLR 2016 (4)
• Continuous Control with Deep Reinforcement Learning, ICLR 2016 (4)
• Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning, ICLR 2016 (2)
• Deep Spatial Autoencoders for Visuomotor Learning, ICRA 2016 (3)
• Learning Deep Neural Network Policies with Continuous Memory States, ICRA 2016 (2)
• End-to-End Training of Deep Visuomotor Policies, JMLR 2016 (2)
• Learning the Variance of the Reward-To-Go, JMLR 2016 (3)
• Deep Reinforcement Learning with Double Q-learning, AAAI 2016 (5)
• Mastering the Game of Go with Deep Neural Networks and Tree Search, Nature 2016 (1)

2015 RL/IL Papers

• Action-Conditional Video Prediction using Deep Networks in Atari Games, NeurIPS2015 (4)
• Gradient Estimation Using Stochastic Computation Graphs, NeurIPS 2015 (1)
• Learning Continuous Control Policies by Stochastic Value Gradients, NeurIPS 2015 (1)
• Deep Attention Recurrent Q-Network, NeurIPS Workshop 2015 (3)
• Deep Recurrent Q-Learning for Partially Observable MDPs, AAAI-SDMIA 2015 (5)
• Trust Region Policy Optimization, ICML 2015 (4)
• Probabilistic Inference for Determining Options in Reinforcement Learning, ICML Workshop 2015 (3)
• Massively Parallel Methods for Deep Reinforcement Learning, ICML Workshop 2015 (2)
• Human-Level Control Through Deep Reinforcement Learning, Nature 2015 (5)

2014 and Earlier RL/IL Papers

• Deep Learning for Real-Time Atari Game Play Using Offline Monte-Carlo Tree Search Planning, NeurIPS 2014 (3)
• Learning Neural Network Policies with Guided Policy Search Under Unknown Dynamics, NeurIPS 2014 (1)
• Deterministic Policy Gradient Algorithms, ICML 2014 (2)
• Playing Atari with Deep Reinforcement Learning, NeurIPS Workshop 2013 (5)
• Tutorial on Linear Function Approximators for Dynamic Programming and Reinforcement Learning, F&T ML 2013 (4)
• A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning, AISTATS 2011 (3)
• Efficient Reductions for Imitation Learning, AISTATS 2010 (3)
• Maximum Entropy Inverse Reinforcement Learning, AAAI 2008 (4)
• Improving Generalisation for Temporal Difference Learning the Successor Representation, N. Computation 1993 (2)
• Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning, M. Learning 1992 (2)
• Active Perception and Reinforcement Learning, N. Computation 1990 (3)

Deep Learning

2019 DL Papers

• On The Power of Curriculum Learning in Training Deep Neural Networks, ICML 2019 (1)

2018 DL Papers

• Stochastic Adversarial Video Prediction, arXiv 2018 (3)
• Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels, NeurIPS 2018 (1)
• Learning to Teach With Dynamic Loss Functions, NeurIPS 2018 (2)
• Skill Rating for Generative Models, arXiv 2018 (3)
• Born Again Neural Networks, ICML 2018 (5)
• Large Scale Distributed Neural Network Training Through Online Distillation, ICLR 2018 (1)
• Learning to Teach, ICLR 2018 (5)
• Interpretable and Pedagogical Examples arXiv 2018 (5)

2017 DL Papers

• Continual Learning with Deep Generative Replay, NeurIPS 2017 (3)
• Attention is All You Need, NeurIPS 2017 (4)
• Dynamic Routing Between Capsules, NeurIPS 2017 (1)
• Tensor Regression Networks, arXiv 2017 (2)
• One Model to Learn Them All, arXiv 2017 (3)
• Population Based Training of Neural Networks, arXiv 2017 (5)
• Automated Curriculum Learning for Neural Networks, ICML 2017 (3)
• Wasserstein GAN, ICML 2017 (3)
• Learning Deep Latent Gaussian Models with Markov Chain Monte Carlo, ICML 2017 (2)
• Get To The Point: Summarization with Pointer-Generator Networks, ACL 2017 (3)
• Adversarial Discriminative Domain Adaptation, CVPR 2017 (4)
• Towards Principled Methods for Training Generative Adversarial Networks, ICLR 2017 (1)
• Unrolled Generative Adversarial Networks, ICLR 2017 (3)
• Understanding Deep Learning Requires Rethinking Generalization, ICLR 2017 (5)
• Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer, ICLR 2017 (2)
• Do Deep Convolutional Nets Really Need to be Deep and Convolutional?, ICLR 2017 (3)
• Overcoming Catastrophic Forgetting in Neural Networks, PNAS 2017 (1)

2016 DL Papers

• NeurIPS 2016 Tutorial: Generative Adversarial Networks, arXiv (4)
• Improving Variational Autoencoders with Inverse Autoregressive Flow, NeurIPS 2016 (1)
• Conditional Image Generation with PixelCNN Decoders, NeurIPS 2016 (2)
• Using Fast Weights to Attend to the Recent Past, NeurIPS 2016 (2)
• Improved Techniques for Training GANs, NeurIPS 2016 (3)
• InfoGAN: Interpretable Representation Learning by Information Maximizing GANs, NeurIPS 2016 (2)
• WaveNet: A Generative Model for Raw Audio, arXiv (1)
• Tutorial on Variational Autoencoders, arXiv (3)
• Active Long Term Memory Networks, arXiv (3)
• Progressive Neural Networks, arXiv 2016 (4)
• Deep Residual Learning for Image Recognition, CVPR 2016 (1)
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning, ICML 2016 (2)
• Visualizing Deep Convolutional Neural Networks Using Natural Pre-Images, IJCV 2016 (1)
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks, ICLR 2016 (2)
• A Note on the Evaluation of Generative Models, ICLR 2016 (3)
• Neural Programmer-Interpreters, ICLR 2016 (1)
• Visualizing and Understanding Recurrent Networks, ICLR Workshop 2016 (1)
• Preconditioned Stochastic Gradient Langevin Dynamics for Deep Neural Networks, AAAI 2016 (3)
• Attention and Augmented Recurrent Neural Networks, Distill (3)

2015 DL Papers

• Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification, ICCV 2015 (2)
• Spatial Transformer Networks, NeurIPS 2015 (4)
• Effective Approaches to Attention-based Neural Machine Translation, EMNLP 2015 (3)
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift, ICML 2015 (4)
• Siamese Neural Networks for One-shot Image Recognition, ICML 2015 (4)
• DRAW: A Recurrent Neural Network For Image Generation, ICML 2015 (2)
• Show, Attend, and Tell: Neural Image Caption Generation with Visual Attention, ICML 2015 (3)
• Going Deeper with Convolutions, CVPR 2015 (1)
• Deep Visual-Semantic Alignments for Generating Image Descriptions, CVPR 2015 (3)
• Very Deep Convolutional Networks for Large-Scale Image Recognition, ICLR 2015 (1)
• ADAM: A Method for Stochastic Optimization, ICLR 2015 (2)
• Explaining and Harnessing Adversarial Examples, ICLR 2015 (2)

2014 and Earlier DL Papers

• Conditional Generative Adversarial Nets, arXiv 2014 (5)
• Recurrent Neural Network Regularization, arXiv 2014 (1)
• Distilling the Knowledge in a Neural Network, DL workshop NeurIPS 2014 (4)
• Generative Adversarial Nets, NeurIPS 2014 (5)
• Recurrent Models of Visual Attention, NeurIPS 2014 (4)
• Visualizing and Understanding Convolutional Networks, ECCV 2014 (3)
• Auto-Encoding Variational Bayes, ICLR 2014 (3)
• On the Importance of Initialization and Momentum in Deep Learning, ICML 2013 (2)
• ImageNet Classification with Deep Convolutional Neural Networks, NeurIPS 2012 (5)
• Large Scale Distributed Deep Networks, NeurIPS 2012 (1)
• Training Deep and Recurrent Networks With Hessian-Free Optimization, NNs Tricks of the Trade, 2012 (1)
• Deep Learning via Hessian-Free Optimization, ICML 2010 (2)
• Curriculum Learning, ICML 2009 (5)
• A Fast Learning Algorithm for Deep Belief Nets, Neural Computation 2006 (1)

Miscellaneous

(Mostly about MCMC, Machine Learning, and/or Robotics.)

2019 Misc Papers

• Picking Towels in Point Clouds, Sensors 2019 (1)

2018 Misc Papers

• Cloth Manipulation Using Random-Forest-Based Controller Parametrization, arXiv 2018 (1)
• Segmenting Unknown 3D Objects from Real Depth Images using Mask R-CNN, arXiv 2018 (4)
• Robot Learning in Homes: Improving Generalization and Reducing Dataset Bias, NeurIPS 2018 (5)
• Establishing Appropriate Trust via Critical States, IROS 2018 (1)
• Learning to See Forces: Surgical Force Prediction w/RGB-Point Cloud Temporal CNNs, MICCAI 2018 (3)
• Manipulating Highly Deformable Materials Using a Visual Feedback Dictionary, ICRA 2018 (1)
• Towards Black-box Iterative Machine Teaching, ICML 2018 (1)
• Dex-Net 3.0: Computing Robust Robot Vacuum Suction Grasp Targets using Analytic Model and DL, ICRA 2018 (4)
• Learning Robust Bed Making using Deep Imitation Learning with Dart, arXiv 2018 (5)
• An Overview of Machine Teaching, arXiv 2018 (5)

2017 Misc Papers

• Machine Teaching: A New Paradigm for Building Machine Learning Systems, arXiv 2017 (5)
• Learning to Fly by Crashing, IROS 2017 (5)
• A Vision-Guided Multi-Robot Cooperation Framework for L-by-Demos and Task Reproduction, IROS 2017 (1)
• Mini-batch Tempered MCMC, arXiv 2017 (3)
• Using dVRK Teleoperation to Facilitate Deep Learning of Automation Tasks for an Industrial Robot, CASE 2017 (4)
• Magnetic Hamiltonian Monte Carlo, ICML 2017 (1)
• Iterative Machine Teaching, ICML 2017 (3)
• Dex-Net 2.0: DL to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics, RSS 2017 (3)
• In-Datacenter Performance Analysis of a Tensor Processing Unit, ISCA 2017 (1)
• Multilateral Surgical Pattern Cutting in 2D Orthotropic Gauze w/DeepRL Policies for Tensioning, ICRA 2017 (5)
• Autonomous Suturing: An Algorithm for Optimal Selection of Needle Diameter, Shape, and Path, ICRA 2017 (4)
• C-LEARN: Geometric Constraints from Demos for Multi-Step Manipulation in Shared Autonomy, ICRA 2017 (3)
• Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection, IJRR (5)
• On Markov Chain Monte Carlo Methods for Tall Data, JMLR 2017 (3)
• A Conceptual Introduction to Hamiltonian Monte Carlo, arXiv (4)
• Model-Driven Feed-Forward Prediction for Manipulation of Deformable Objects, IEEE TASE 2017 (3)

2016 Misc Papers

• SWIRL: A Sequential Windowed IRL Algorithm for Robot Tasks With Delayed Rewards, WAFR 2016 (2)
• Minimum-Information LQG Control Part I: Memoryless Controllers, CDC 2016 (2)
• Minimum-Information LQG Control Part II: Retentive Controllers, CDC 2016 (1)
• Adaptive Optimal Training of Animal Behavior, NeurIPS 2016 (3)
• Cooperative Inverse Reinforcement Learning, NeurIPS 2016 (3)
• Bayesian Optimization with Robust Bayesian Neural Networks, NeurIPS 2016 (2)
• Tumor Localization using Automated Palpation with Gaussian Process Adaptive Sampling, CASE 2016 (3)
• Robot Grasping in Clutter: Using a Hierarchy of Supervisors for Learning from Demonstrations, CASE 2016 (4)
• Gradient Descent Converges to Minimizers, COLT 2016 (3)
• Scalable Discrete Sampling as a Multi-Armed Bandit Problem, ICML 2016 (1)
• Supersizing Self-Supervision: Learning to Grasp from 50K Tries and 700 Robot Hours, ICRA 2016 (5)
• Dex-Net 1.0: A Cloud-Based Network of 3D Objects for R. Grasp Planning Using a M-A Bandit w/CR, ICRA 2016 (4)
• TSC-DL: Unsupervised Trajectory Segmentation of Multi-Modal Surgical Demonstrations with DL, ICRA 2016 (3)
• Automating Multi-Throw Multilateral Surgical Suturing with a Mechanical Needle Guide and SCO, ICRA 2016 (4)
• Supervised Autonomous Robotic Soft Tissue Surgery, Science Translational Medicine, 2016 (3)

2015 Misc Papers

• Bayesian Dark Knowledge, NeurIPS 2015 (2)
• A Complete Recipe for Stochastic Gradient MCMC, NeurIPS 2015 (2)
• Large-Scale Distributed Bayesian Matrix Factorization using Stochastic Gradient MCMC, KDD 2015 (1)
• The Fundamental Incompatibility of Scalable Hamiltonian Monte Carlo and Naive Data Subsampling, ICML 2015 (2)
• LbO Surgical Subtasks: Multilateral Cutting of 3D Viscoelastic and 2D Orthotropic Tissue Phantoms, ICRA 2015 (4)

2014 Misc Papers

• Eliciting Good Teaching From Humans for Machine Learners, Artificial Intelligence 2014 (4)
• Bimanual Telerobotic Surgery With Asymmetric Force Feedback: a da vinci Implementation, IROS 2014 (3)
• Learning Accurate Kinematic Control of Cable-Driven Surgical Robots Using Data Cleaning and GPR, CASE 2014 (5)
• Austerity in MCMC Land: Cutting the Metropolis-Hastings Budget, ICML 2014 (4)
• Stochastic Gradient Hamiltonian Monte Carlo, ICML 2014 (4)
• Hamiltonian Monte Carlo Without Detailed Balance, ICML 2014 (2)
• Towards Scaling up Markov Chain Monte Carlo: An Adaptive Subsampling Approach, ICML 2014 (4)
• Autonomous Multilateral Debridement with the Raven Surgical Robot, ICRA 2014 (5)
• Teaching People How to Teach Robots, HRI 2014 (4)
• RRE: A Game-Theoretic Intrusion Response and Recovery Engine, IEEE Trans on P&D Systems 2014 (4)

2013 and Earlier Misc Papers

• A Case Study of Trajectory Transfer Through Non-Rigid Registration for a Simplified Suturing Scenario, IROS 2013 (2)
• Finding Locally Optimal, Collision-Free Trajectories with Sequential Convex Optimization, RSS 2013 (3)
• Learning Task Error Models for Manipulation, ICRA 2013 (4)
• Training a Robot via Human Feedback: A Case Study, ICSR 2013 (1)
• Bayesian Posterior Sampling via Stochastic Gradient Fisher Scoring, ICML 2012 (2)
• Algorithmic and Human Teaching of Sequential Decision Tasks, AAAI 2012 (4)
• The Jacobian Condition Number as a Dexterity Index in 6R Machining Robots, RCIM 2012 (2)
• Bayesian Learning via Stochastic Gradient Langevin Dynaimcs, ICML 2011 (4)
• Bringing Clothing Into Desired Configurations with Limited Perception, ICRA 2011 (3)
• Visual Measurement of Suture Strain for Robotic Surgery, C&MM in Medicine 2011 (3)
• Towards an Assistive Robot that Autonomously Performs Bed Baths, IROS 2010 (4)
• Development of a Nursing-Care Assistant Robot RIBA, IROS 2010 (4)
• Cloth Grasp Point Detection Based on Multiple-View Geometric Cues w/Applic. to Towel Folding, ICRA 2010 (5)
• MCMC Using Hamiltonian Dynamics, Handbook of Markov Chain Monte Carlo 2010 (4)
• Active Perception: Interactive Manipulation for Improving Object Detection, Technical Report 2010 (3)
• Superhuman Performance of Surgical Tasks by Robots using Iterative Learning from H-G Demos, ICRA 2010 (2)
• Active Learning for Real-Time Motion Controllers, SIGGRAPH 2007 (3)
• Core Knowledge, Developmental Science 2007 (3)
• Effect of Sensory Substitution on Suture-Manipulation Forces for Robot Surgery, J Thorac Cardiovasc Surg 2005 (3)
• Analysis of Suture Manipulation Forces for Teleoperation with Force Feedback, MICCAI 2002 (3)
• An Introduction to the Conjugate Gradient Method Without the Agonizing Pain, Technical Report, 1994 (3)
• Active Perception, Proceedings of the IEEE 1988 (2)

paper_notes's Issues

The Implementation of Guided Cost Learning

Dear Dr. Seita,

Thanks for your paper_notes. I am now working on an autonomous driving project based on GCL. However, I have no idea how to implement the GCL algorithm. I really want to implement the GCL algorithm, I was wondering if you could provide some implementation code with me. Thank you.

Sincerely

About the code!

Hello, I am reading papers about the "learning to learn",I have some doubts about the implements of RL2, just like one trail includes n same MDPs or n different MDPs, or the meaning of fast and slow,even the difference of it and MAML.You mentioned you would read the code,could you email the code to me for my research work!I haven't found any code about RL2! I will appreciate your help if you can.

on what “representation” means

Hello, and thanks for your very insightful review to many papers I'm deeply interested in.

In the "Loss is its own reward" paper's review, you said:

They're interested in representation, which in English, means how we encode and communicate "good-ness" and "bad-ness" to an agent

In this sentence, two parts of the model is implied. An unamed part encodes and communicates the "good-ness", and agent. However, classifying which action was taken between two frames, can be done without the knowledge of what the agent values, or really, any reward at all. And classifying which action was taken is a self-supervisation task that is used to improve the representation. (I'm sure you already understand and agrees with all this, just bringing my reasons to your attention)

Would it not be more accurate to say representation is about learning the underlying mechnisms of the environment, akin to visual development and intrinsic physics in infants, which would (hopefully) help the agent to decide "good-ness" and "bad-ness" of a state?