Uninterpreted streams of observations and commands from a robotic sensorimotor cascade.
My dissertation is about the bootstrapping problem: learning a model of a robot's sensorimotor cascade, and its environment, from zero prior information other than basic semantic. At the beginning, we only assume to have uninterpreted streams of bits representing the output of some sensors, and that we have some commands available, and that there is some causal relation from actions to observations. The situation is very similar to the video on the right.
The video shows learning of a bilinear model of a sensorimotor cascade for a camera. The agent starts with no previous knowledge on the sensor geometry, and by correlating observations with commands, it can learn a generative model for the data. The same model can be used for learning the dynamics of different sensors (range-finder, camera, field sampler). See many other videos of related experiments.
Other than a worthy problem per se (it subsumes most problems of learning, calibration, fault detection, etc.), it is also a proxy for studying some aspects of the higher level of neural processing. In fact, it is believed that the cortex starts as a tabula rasa that adapts to the inputs (the evidence is that parts of it can be repurposed in subjects that lost some sensory capacity).
At this point, it is not clear if an agent can learn everything from the environment, or if there is something that should be known a priori. My goal has been to try to derive some precise formulation of the problem, and to find strong results (in the spirit of control theory) that can be built upon.
Dissertation
Bootstrapping Vehicles: a formal approach to unsupervised sensorimotor learning based on invariance.
Technical Report, California Institute of Technology, 2012.
pdf
supp. material
slides
Could a "brain in a vat" be able to control an unknown robotic body to which it is connected, and use it to achieve useful tasks, without any prior assumptions on the body's sensors and actuators? In this work, the problem of "bootstrapping" is studied in the context of the Vehicles universe, which is an idealization of simple mobile robots, after the work of Braitenberg. The first thread of results consists in analyzing such simple sensorimotor cascades and proposing models of varying complexity that can be learned from data. The second thread regards how to properly formalize the notions of "absence of assumptions", as a particular form of invariance that the bootstrapping agent must satisfy, and proposes some invariance-based design techniques.
This is version 1.3 of my dissertation -- after turning it in, I am still working to integrate the newer material that I had not the time to add. So any comments/suggestions are welcome!
Recent papers on boostrapping
Motion planning in observations space with learned diffeomorphism models.
In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2860–2867. Karlsruhe, Germany, 5 2013.
pdfdoi supp. material
Using learned diffeomorphism models of the dynamics of cameras and range-finders, we formulate motion planning as a planning problem in the observations space. Nodes/states are (uncertain) images; actions/edges are (uncertain) diffeomorphisms.
Calibration by correlation using metric embedding from non-metric similarities.
IEEE Transactions on Pattern Analysis and Machine Intelligence, 35:2357–2370, 10 2013.
pdfdoi supp. material
Davide and I explain how to calibrate a generic single-view-point camera by only waving it around.
paper finalist)
Fault detection and isolation from uninterpreted data in robotic sensorimotor cascades.
In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA). Saint Paul, MN, May 2012.
pdfdoi supp. material slides
The following are already included in my dissertation:
Learning diffeomorphism models of robotic sensorimotor cascades.
In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA). Saint Paul, MN, May 2012.
pdfdoi supp. material slides
We show that diffeomorphisms can represent the dynamics of both range-finders as well as cameras, and can be easily learned from raw sensorimotor data. A follow-up paper shows how to do planning in observations space with these learned diffeomorphisms.
Uncertain semantics, representation nuisances, and necessary invariance properties of bootstrapping agents.
In Joint IEEE International Conference on Development and Learning and Epigenetic Robotics. Frankfurt, Germany, August 2011.
pdfdoi slides
This paper tries to describe formally the idea of "uncertain semantics" in a bootstrapping problem by using the mathematics of group actions. If interested, please see Part 1 and Part 3 of my dissertation, where these ideas are further expanded and made much more formal than in this short paper.
Bootstrapping sensorimotor cascades: a group-theoretic perspective.
In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). San Francisco, CA, September 2011.
pdfdoi supp. material
This paper discusses a class of models (BGDS) that capture the bilinear dynamics of heterogeneous sensors, such as field samplers, cameras, and range-finders, as well as their dependence on the gradient of the observations.
This material has been greatly expanded in my dissertation (Part 2).
paper finalist)
Bootstrapping bilinear models of robotic sensorimotor cascades.
In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA). Shanghai, China, May 2011.
supp. material slides
This paper shows that simple bilinear models capture the dynamics of heterogeneous sensors, such as field samplers, cameras, and range-finders. These models can be learned unsupervisedly and used to perform simple tasks.
This material has been greatly expanded in the following paper.