GithubHelp home page GithubHelp logo

ammarkov / robovision Goto Github PK

View Code? Open in Web Editor NEW
22.0 7.0 10.0 186.63 MB

Attempting to create a program capable of combining stereo video input , with motors and other sensors on a PC running linux , the target is embedded linux for use in a robot!

Home Page: http://ammar.gr/gddg

C++ 29.88% C 55.78% Shell 2.44% MATLAB 0.28% Processing 1.73% Python 0.51% PHP 0.25% CMake 0.02% HTML 8.94% Cuda 0.18%
robot-vision robotics software c computer-vision linux embedded arduino stereo-vision stereo-camera

robovision's Introduction

AmmarServer

RoboVision

A GPLv3 Computer Vision / Robotics Project that aims to provide a compact software stack for generic vision enabled robotics!

Documentation


To start , you can begin by reading my BSc thesis which while give you a detailed overview and the computer vision theory that it is based on in the Documentation directory.

https://github.com/AmmarkoV/RoboVision/blob/master/Documentation/GuarDDoG_RoboVision.pdf?raw=true ( or here http://ammar.gr/GuarDDoG_RoboVision.pdf ) ( or here http://users.otenet.gr/~ammarkov/files/GuarDDoG_RoboVision.pdf )

or the presentation slides

https://github.com/AmmarkoV/RoboVision/blob/master/Documentation/guard_dog_presentation.pdf?raw=true ( or here http://ammar.gr/guard_dog_presentation.pdf ) ( or here http://users.otenet.gr/~ammarkov/files/guard_dog_presentation.pdf )

My more recent work is increasingly human/object tracking relevant and you can see an updated list of publications here https://scholar.google.gr/citations?user=sDOdhtwAAAAJ&hl=en&oi=ao

Status


I have stopped active development on this project for several years now, since I am currently working in EU robotics projects ( http://hobbit.acin.tuwien.ac.at/ , http://www.ramcip-project.eu/ramcip/ and others )

This work is still premature but in the future maybe I will revisit it and continue to work on it. The testing platform for the software is my pet project named “GuardDoG” ( http://ammar.gr/gddg/ ) , a surveillance robot guard for homes and offices. Most of the layers of the project are abstract enough to be able to be used by different designs.

If you don't have the specific motors/ sensors etc , this repository will not be able to work out of the box, you can however use specific parts of it since isolated they might be useful.

Spin-offs


Parts of this original work have now been split in seperate "less monolithic" spin-off repositories , such as

https://github.com/AmmarkoV/RGBDAcquisition - A uniform library wrapper for input from libfreenect,OpenNI,OpenNI2,OpenGL simulations,V4L2 and other types of video and depth input.. This basically is a much better version of the "Video Input" module of this repository which is transparently compatible with a large variety of sensors including RGBD cameras.

https://github.com/AmmarkoV/AmmarServer - A lightweight http server for linux This can help in building a proper web interface ( instead of the simple one in RoboKernel/webinterface.c which relies in files shared between this and an Apache session )

https://github.com/AmmarkoV/OpenDroneA - A repository for the blueprint of a quadcopter drone A drone can be an alternative "body" for a robot vision system

https://github.com/AmmarkoV/AdvancedImageSearch - Command line tools that enable content-aware image browsing from the CLI

Implementation


RoboVision works ONLY on linux, and uses V4L2, wxWidgets, OpenGL, Festival, CMU Sphinx, OpenCV , Apache and other foss as extensions. I have made an effort to implement everything in a low enough level so that the console “daemon” mode will need only V4L2 support, and everything will be compiled together to reduce unneeded overheads. Lately I have added OpenCV as a library dependency to speed up development by using some thoroughly tested function calls and reducing the debugging surface. An effort has been also made to keep libraries clean of bloat and the whole project could easily be ported to another platform , but there is currently no benefit for the extra maintenance cost , so there are no ( and probably will never be :P ) plans for porting it to Windows or Mac.

Due to the complexity and combination of computer science fields involved ( Computer Vision , Natural Language Processing , Artificial Intelligence , Embedded Systems , Robotics [among the most important] ) the project has still not reached maturity and development is continued until an acceptable functionality level is reached..

Once you download the project ..

To automatically resolve the dependencies of the project run .Scripts/Setup/apt-get-guarddog-dependencies if you have a Debian/Ubuntu based distro ( package names may vary from version to version , but I try to update them to the last possible version ) , or using .Scripts/Setup/pacman-guarddog-dependencies for Arch and pacman based distros..

To compile it you will need to run the custom make bash script in the parent directory typing ./make from the shell while being on the root directory of the project ( NOT "make" BUT "./make" it is not a standard makefile but a bash script )

The IDE used for the development of the Project is Code::Blocks , and you can use the workspaces on the root directory that open the sub projects , in any case , once you download a fresh copy of the project be sure to run ./make first to compile any libraries that may be needed and are not part of the workspace

I Hope you'll find this project/repository useful ..!

robovision's People

Contributors

ammarkov avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

robovision's Issues

AuditoryInput

Add speech/sound recognition ( i.e. CMU Sphinx ) ..

Good things :
1 ) Performance
2 ) Stereo Mics ( use both microphone and line-in to estimate position of sound source )
3 ) Quality
4 ) Non Human sounds identification ( i.e. Moo -> Cow.. )
5 ) Greek language :P

Add a local Artificial Intelligence knowledge base stack with a little NLP

Add an Artificial Intelligence stack with a little NLP ( Natural Language Processing ) ..
GuarddoG already does that inside RoboKernel , as the commands it executes are "strings" that have been mapped to functions as seen here ( https://github.com/AmmarkoV/RoboVision/blob/master/RoboKernel/command_hal.c ) ..
The placeholder for such functionality is https://github.com/AmmarkoV/RoboVision/tree/master/RVKnowledgeBase which should also contain facts about the world in order for a command to be able to have a syntax of "Go to the kitchen" , and guard dog to be able to respond to the command ..

VisualCortex/SLAM

An effective video input based SLAM algorithm must be implemented int VisualCortex

Add MMX/SSE2 assembly inline

The Visual Cortex functionality can be dramatically increased by fully using the Intel MMX/SSE2+ instruction set since it offers specific low level commands for executing in parallel the operations that are now sequential and are left to gcc to be optimized..!

Add an online knowledge base backbone to the local knowledge base

Add a query system utilizing Wolfram Alpha , or Open Mind or Google or something big :P , in order to process questions such as
"How old is jennifer lopez ? " ?
Wolfram Alpha answers very acurately with a "42 years 2 months 11 days" which would be great functionality for guarddog!

Remove OpenCV dependencies..

From the start of the project the decision was not to use generic functions but make an optimized pipeline for stereoscopic vision.. I have done that but to speed up development I use some OpenCV functions , namely for HaarCascades and Homography estimations , with a little work they can be discarded..

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.