Automated Object Recognition Technology with Machine Learning
Content of this page
Why SentiSight?
Neurotechnologija's SentiSight technology is intended for developers who want to use computer vision-based object recognition in their applications. SentiSight enables the learning of objects and searching for learned objects in images from almost any camera, webcam, still picture or live video.
- Universal. The SentiSight algorithm is designed to be as universal as possible. It can support web cameras, surveillance cameras and can input images from the picture. It is tolerant to object scale, rotation, pose etc. Some of the potential applications for SentiSight technology include:
- Search engines that recognize objects in picture files (either local or on the Web);
- Security systems;
- Parts recognition in production lines;
- Robot vision;
- Road signs recognition;
- Machine vision.
- Fast. SentiSight can process video streams in real time, so it can be used for real-time applications.
- Webcam capable. Though high quality cameras will provide better recognition quality, a simple webcam is enough for SentiSight operation.
- Flexible licensing model and competitive price.
Algorithm
|
The SentiSight object recognition algorithm implements advanced visual-based object learning and recognition.
|
1.) Object Learning
|
In order to recognize an object in an image, the appearance of an object must first be memorized. In the learning phase, SentiSight algorithms extract specific object features from a video stream or single image and save them into what is known as a model template.
In many cases there is more information in a video or single image than just the object you want SentiSight to learn, like a background, other objects in the room or a hand holding the object. For this reason, certain steps should be taken during the "learning" process to provide information about the exact location of the desired object in the image. This should be done with a mask of the object in the image. A mask explicitly specifies which pixels of the image belong to the object and which ones are part of the background. Thus, only object-specific information will be included into the model template.
If there is no way to provide a mask for the image, SentiSight can still learn the object. However the other background elements would be learned togethIr with the object. This can affect the ability of the algorithm to recognize the unique qualities of the object and may result in the object being misclassified with other objects that have the same background.
However, for a lightweight movable object SentiSight does provide a fully automatic learning procedure. To learn a lightweight movable object in the SentiSight SDK, the user should do following steps:
- Choose a static background (preferably smooth) and direct the camera to it.
- Choose a holder – an object that will be used to hold and move the learned object. A user's hand can be the "holder".
- This "holder" should be presented it to the camera first, in various poses and configurations (if it is not rigid object) so that it can be learned by SentiSight.
- After the holder has been learned, SentiSight is ready to learn the object itself, by having the holder rotate and move the object closer and further from the camera.
- Input the learned object name (ID) into the system.
The process is somewhat different for objects that cannot be moved or if only images of the objects are available. In these cases, one should set the mask of the object manually or do not provide a mask at all.
|
2.) Recognition
|
For the recognition of the object, the camera is directed to the scene where the learned object is presented or may appear. No other action is required. When the object appears in the vision field, it is recognized by SentiSight, which outputs the object's name (ID) and coordinates.
The SentiSight algorithm is tolerant to large variations of object scale, object rotation and translation. When an object is learned, the algorithm creates a model with possible views from different sides, in different 3D poses and in different lighting conditions.
SentiSight's object recognition is comparably fast – around 10 frames per second for a single object model (320 X 240 resolution). However for tasks when an even faster response is needed, the SentiSight Library has a tracking mode that enables tracking speeds up to 20 frames per second. Tracking is initialized if an object is recognized and located, then tracks the object until it changes somewhat in appearance, at which point tracking is reinitialized by recognition. The tracking feature is sensitive to complex backgrounds, and tracking is more difficult with homogenous objects
All performance evaluations were performed using a PC with 2.4 GHz Intel Core2 Duo CPU
|
|
Background learning
 Click to zoom
"Holder" learning
 Click to zoom
Object learning
 Click to zoom
|
Object recognition
 Click to zoom |
Reliability Tests and Technical Specifications
SentiSight was tested with object images from many cameras. At 0.1% False Acceptance Rate (FAR), the recognition rate is from 70% to more than 99% depending on object structural appearance, transparency, etc. For objects with well defined intenal structure, the recognition rate is 98% - 99% at 0.1% FAR.
| SentiSight algorithm technical specifications |
| Recommended image size for real time operation on modern processor |
320 x 240 pixels |
Static Background Extraction/Object mask separation
(320 x 240 image size) |
20 frames/sec |
Learning: Processing of single objects' frame
(320 x 240 image size) |
0.05 sec. |
Learning: Generalization time
(for 100 frames of object) |
6 sec. |
Recognition speed from image frame for single object model
(including processing of the image) |
~ 10 frames/sec. |
Recognition speed from image model for single object model
(excluding processing of the image) |
~ 20 models/sec |
All performance evaluations were performed using a PC with 2.4 GHz Intel Core2 Duo CPU
System Requirements
- PC with 1GHz processor supporting SSE2 technology;
- 256 Mb of RAM.
Algorithm Demo
|
|
The SentiSight demo application for Microsoft Windows 2000/XP/2003/Vista can be downloaded for evaluation of the SentiSight vision based object recognition algorithm. The application learns and recognizes objects from almost any camera or webcam, image and video files. Internet connection is not required to run the application.
SentiSight SDK trial is also available for downloading.
|
|
Related Products
SentiSight Standard SDK is based on SentiSight technology.
|
Click to zoom 
Click to zoom 
Click to zoom 
Click to zoom 