On noise, parameter identification, and Jacobian estimation

Apparently, noise benefits system identification. Measurement noise, however, if more than a couple of pixels, ruins the Jacobian estimation. It's not rocket science, it's all obvious. But after artificial generation of all of the visual-motor experiments and verifying the correctness of the Jacobian initialization method, I can sleep better at night. The wrong estimation of the visual-motor Jacobian was only due to the visual tracking noise and one faulty recorded data.

For the MATLAB simulations that verified the above, I calibrated the camera (COSMICAR 6mm Lens, PointGrey Dragonfly Express Firewire Camera) with the Camera Calibration Toolbox. Then I created a camera and placed it on the WAM arm model that I'd created earlier using the Robotics Toolbox and guessed the 3D coordinates of the feature points that I placed on a virtual wall in the simulations. Since the joint angle readings from the WAM arm are very accurate, I used the exact same readings and projected the feature points on the camera image at the new camera poses. I noticed that when using a few data points, adding a zero-mean Gaussian noise with a very small variance reduces the error in the estimation of the next joint value. When I increased the variance, the estimation became erroneous, as expected. For example, the results were reliable when the standard deviation of the noise was in the order of 0.05, but when the standard deviation was 1.5, then the estimation error was very large.

To avoid the unavoidable measurement noise, I'd filter the visual tracking results every 4-5 readings. Before filtering, the system tracks at 100 Hz, and I guess I can sacrifice some bandwidth for now (the system still works in 20-25 Hz).

{discussion.vision} What can a machine see?

A general question is posed as the topic of this entry. There are different approaches to solve different problems in computer vision. I am specifically curious to know what problems each different approach in computer vision solves and how much a priori information each approach considers. I also like to know how far we are from replicating a biological system, in your opinion. Are we in the right direction to do so in computer vision?

The classification of the different approaches are not quite clear (see SoloGen's* comment on the previous entry), but we might trivially try to categorize them into (a) biological and (b) non-biological (artificial) vision. You may wonder why I am considering biological vision when writing on computer vision. I have two reasons: (1) nature is man's best teacher, and (2)artificial gadgets can be integrated in biological visual systems to correct/improve malfunctions (e.g., see Progress in Artificial Vision). Because of these two reasons, many are investigating how biological visual systems work. Having all these said, I will leave biological vision for another discussion and focus on non-biological vision.

Based on how much we know about the sensor (camera), non-biological vision can be sub-categorized to (b-1) calibrated, where parameters of the camera are known and (b-2) uncalibrated, where they are not known. We can also categorize them further by knowing about the world (how much we know about the scene model, and what type of images we are dealing with).

In this sense, we can assume that Geometrical approaches assume that the type of the camera is known and the scene model can be represented geometrically. For example, there is a calibrated projective camera and the scene consists of boxes of a solid texture.

Learning-based approaches step farther and assume that there will be no a priori knowledge of the scene model and everything is to be learned. It still makes sense if we know something about the camera, as without any a priori knowledge we will be looking at a near-infinite space of pluasible inferences for a scene.

I think this would start off the discussion. Feel free to elaborate more on these categories or propose others while we continue the discussion.

You can either write in the comments section, or if you prefer, you can write in your own blog and I link to you in the next entry.


P.S. I don't believe much in discussion forums. A blog is a better place to write and exchange ideas, I believe. I intend to summarize our ideas and prepare an online version for future reference.

---
* SoloGen maintains two blogs: Thesilog and Anti Memoirs.

{who.cogSci} Stevan Harnad

Stevan Harnad is renowned cognitive scientist [wiki] and author of the following article:

Harnad, S. (1994) Computation Is Just Interpretable Symbol Manipulation: Cognition Isn't. Special Issue on "What Is Computation" Minds and Machines 4:379-390
[Full Text HTML]

This post was originally meant to introduce a scientist and one of his works that interested me. Though I am not particularly working on cognition, my research would generally be on computational vision and its applications to robotic visual servoing, i.e., vision-based control of robots. I have been passionate about the geometrical/computational approach to 3D vision, but I have to be aware of the limitations of such an approach, as the boundary between biological vision and cognition is blurred.

I have had a couple of interesting discussions with SoloGen on the issue. I could re-phrase the issue as "Which would ultimately make a better artifical visual system for daily/industrial applications: Machine Learning or Geometry?"

I view computational vision is an intricate and interesting problem, but the question is, does solving such a problem help humans towards building their intelligent clones? I must point out that, in my opinion, humans are subconciously (?) in search of making their own version of human-like machines, but are too conservative to boldly challenge their creator. Afterall, it would be more likely that the contemporary scientists fail to make the artificial thinking/intelligent human, so why enter a lose-lose competition?

Master of Science in Vision and Robotics

Here is a list of courses that I think would give the right background (mathematics, computer science, mechanics, signal processing) to do good work in computer vision and robotics. Probably good for a 5-year course based master's degree in an EECS department. The more I work in this area, the more I feel that Robotics and Computer Vision is a field by itself and can't fit the traditional EE or CS program, because it needs a strong math background, a solid and understanding of mechanics in addition to the traditional education given in EE and CS. For example, many courses in traditional EE are not useful (for example 3 courses in electronics) to robotics and computer vision in particular. The CS graduates don't traditionally learn about signal processing and control theory on the other hand. We should come up with a good mixture.

--------------------------
Math background
--------------------------
Logic
Calculus I, II
Real Analysis
Linear Algebra
Algebraic Geometry
Differential Algebra
Complex Analysis
Ordinary Differential Equations
Partial Differential Equations
Applied Probabilities and Statistics
Numerical Methods for Computation
Graph Theory
Complexity Theory
Mathematical Optimization
Differential Geometry
Variational Calculus
-- 18

---------------------------------------
Computer Science background
---------------------------------------
Algorithms I,II
Complexity Theory
Languages and Automata
Programming
Object-oriented programming
UNIX programming
Computer Organization
Operating Systems
Formal Methods
System Architecture
System Validation and Verification
-- 12

--------------------------------------
Signal Processing Background
--------------------------------------
Fourier Analysis
Signals and Systems
Information Theory
Digital Signal Processing
Digital Image Processing
-- 5
---------------------------------------
Computer Vision and Graphics
---------------------------------------
Foundations of Computer Vision
3D Computer Vision
Feature Extraction and Motion Tracking
Computer Graphics
--4

--------------------------------------------------
Control Theory and Machine Learning
--------------------------------------------------
Linear Control Theory
Digital Control Theory
Nonlinear Control
System Identificaton
Statistical Methods in Learning
Machine Learning
Optimal Control
-- 7

--------------------------------
Mechanics and Robotics
--------------------------------
Dynamics
Statics
Digital Design
Mechatronics (Microcontrollers, Servo motors, DC motors, necessary electronics)
Robot Mechanics
Mobile Robots and Navigation
Calibration Methods
-- 7

50 courses

{who.compVis} Joseph Mundy

Joseph Mundy is a pioneer computer vision scientist, who observed the importance of Algebraic Geomtery in computer vision some 20 years ago. The modern approach to computer vision uses Projective Geometry as the mathematical framework to represent and analyse scenes. Projective Geometry is an established field of Projective Spaces within the Algebraic Geometry community. It appears in graduate course textbooks (such as Hartley and Zisserman's Multiple View Geometry in Computer Vision) and will soon find its way to the undergraduate curriculum. Here is the wiki page for Mundy (includes a quote that highlights the important fields related to computer vision).

Speaking of formal education required for the field of computer visoin, my expectation is that the next generation of (non-traditional) computer science curricula will include more Algebra and Geometry. In fact, it is about the time where a new computer vision discipline emerges that covers all the areas related to image analysis and computer vision. Currently, CS students do not learn Fourier Analysis and Systems Theory, nor do they learn Algebra and Geometry. A CS graduate is not prepared to carry on research in computer vision. Similarly, a classical robotics guy specializes in Robot Mechanics (kinematics and dynamics) and may learn some working knowledge of motors (step motors, dc motors, etc.) from Electrical Engineering department. If a hardcore computer vision researcher wants to be involved in robotics (mobile robotics and industrial robotics), he will need a lot more than the current conventional courses.

{science.vision} CVR 2005

I will be attending the Computational Vision in Neural and Machine Systems conference held in York University form tomorrow till Saturday.

It is my first visit to York University. There would be invited talks from renowned professors in vision research with two flavours: computer vision and neuroscience.

Berkeley's Jitendra Malik, Duke's Carlo Tomasi, U of T's Allan Jepson are only those whose work I have seen before. I am sure if you are into neuroscience, you will find many interesting talks by those who know what they are saying. And this is very important. Very few people know what they are saying and I don't like to miss the opportunity to listen to them (so I should go and get some sleep now!).

BTW, if you want to pick on me on why I have omitted the titles Dr./Prof. from the names of these scientists, I should emphasize that when you believe in someone's work, you will never give him a title. The name alone says it all. Have you ever heard someone say Professor Newton, or Professor Maxwell?