Object Trackinng PHd Thesis

  • Published on
    21-Apr-2015

  • View
    153

  • Download
    2

Embed Size (px)

Transcript

Master Erasmus Mundus in Color in Informatics and Media Technology (CIMET)

ObjectTracking:StateofTheArtandCAMSHIFTImprovementUsing MultidominantColorsTracking MasterThesisReport Presentedby

PriyantoHidayatullah anddefendedatthe

UniversityofJeanMonnetSaintEtienne,France nd 22 June2010 JuryCommittee: Prof.AlainTremeau Prof.JonYngveHardeberg FaouziAlayaCheikh,Ph.D JavierHernndezAndrs,Ph.D DamienMuselet,Ph.D EricDinet,Ph.D Supervisor: HubertKonik,Ph.D

Object Tracking: State of The Art and CAMSHIFT Improvement Using Multi-dominant Color Tracking

Abstract

Object tracking is a wide area in which a lot of methods available and wide variety of applications. One of the applications would be tracking an object in a clickable hypervideo to enrich the interactivity of video application. In this thesis, some state of the art of object tracking methods are reviewed and closely observed. We then select one of object tracking state of the art methods to improve. Our selection goes to CAMSHIFT which has been very well accepted as one of the most prominent methods in object tracking which has real time speed performance and more suitable for clickable hypervideo. CAMSHIFT is very good for single hue object tracking and in the condition where objects color is different with backgrounds colors. In this thesis, we try to improve the robustness of CAMSHIFT for multihued object tracking and the situation where objects colors are similar with backgrounds colors. To improve robustness on the condition where objects colors are similar to backgrounds colors, we use object localization by selecting each dominant color object part using combination of Mean-Shift segmentation and region growing. Hue-distance, saturation and value color histogram are used to describe the object. We also track the dominant color object parts separately and combine them together to improve robustness of the tracking on multihued object. Our experiments showed that those methods improved CAMSHIFT significantly. This improvement hopefully will be useful for object tracking in clickable hypervideo. Keywords: Object tracking, CAMSHIFT, Segmentation, Mean-Shift, Hypervideo.

i

Object Tracking: State of The Art and CAMSHIFT Improvement Using Multi-dominant Color Tracking

Table of ContentsAbstract ...................................................................................................................... i Table of Contents ....................................................................................................... ii Table of Figures ......................................................................................................... iv 1 1.1 2 2.1 2.2 Introduction ...................................................................................................... 1 The General Aim of The Master Thesis ....................................................... 1 Previous Work ................................................................................................... 2 Test Videos ................................................................................................ 2 Object Tracking Categorization .................................................................. 4

2.3 Corner Detector Combined with Optical Flow ............................................ 4 2.3.1 Corner detection .................................................................................... 5 2.3.2 Optical flow ........................................................................................... 5 2.4 2.5 Speeded Up Robust Features (SURF) ......................................................... 7 Mean Shift Tracking .................................................................................. 9

2.6 CAMSHIFT Tracking ................................................................................ 11 2.6.1 Color probability distribution and histogram back projection ............... 12 2.6.2 Mass center calculation ....................................................................... 13 2.6.3 CAMSHIFT advantages and disadvantages .......................................... 14 2.7 2.8 2.9 Local Binary Pattern ................................................................................ 15 Beyond Semi-Supervised Online Boosting Tracking ................................. 17 Method that We Choose ........................................................................... 19

2.10 CAMSHIFT/Mean-Shift Improvement in Literatures ...............................20 2.10.1 Mean-Shift tracking combined with texture histogram.....................20 2.10.2 CAMSHIFT and Mean-Shift combined with interest points .............. 21 2.10.3 CAMSHIFT improvement using new HSV model ............................. 24 2.10.4 CAMSHIFT improvement using hue-distance and saturation features25 2.10.5 CAMSHIFT with improvement of object localization........................ 27 2.10.6 CAMSHIFT improvement using adaptive background (ABCShift) .... 28 2.10.7 CAMSHIFT improvement by background subtraction...................... 31 2.10.8 The CAMSHIFT improvement method that we choose ..................... 32 2.10.9 The more specific aim of the master thesis ....................................... 32 3 Proposed Method ............................................................................................. 34 3.1 Object Localization .................................................................................. 34 3.1.1 Preprocessing ...................................................................................... 34 3.1.2 Image color transformation ................................................................. 36 3.1.3 Object Selection................................................................................... 36 3.1.4 Minimum and maximum values storing ............................................... 36 3.2 3.3 Object Modeling ...................................................................................... 37 Making Color Mask.................................................................................. 38

ii

Object Tracking: State of The Art and CAMSHIFT Improvement Using Multi-dominant Color Tracking

3.4 3.5 3.6 4 4.1 4.2 5

Segmentation .......................................................................................... 38 Histogram Back Projection ...................................................................... 39 Tracking .................................................................................................. 39 Implementations ..................................................................................... 42 Experiments Setting ................................................................................ 43

Implementations and Experiments .................................................................. 42

Results and Discussions ................................................................................... 44 5.1 Results .................................................................................................... 44 5.1.1 First Experiment Results ..................................................................... 44 5.1.2 Second Experiment Results ................................................................. 46 5.1.3 Third Experiment Results.................................................................... 49 5.1.4 Forth Experiment Results.................................................................... 49 5.2 Discussion ............................................................................................... 52 5.2.1 Some Advantages ................................................................................ 52 5.2.2 Some Limitations ................................................................................ 52

6

Conclusions and Future Works ........................................................................ 54 6.1 6.2 Conclusions ............................................................................................. 54 Future Works .......................................................................................... 54

7

Bibliography .................................................................................................... 55

iii

Object Tracking: State of The Art and CAMSHIFT Improvement Using Multi-dominant Color Tracking

Table of FiguresFigure 2.1 Test Videos. ............................................................................................... 2 Figure 2.2 Illustration of optical flow[28]. .................................................................. 6 Figure 2.3 Shi Tomasi corner detector also detect the background corners inside objects rectangle .................................................................................................................... 7 Figure 2.4 SURF Tracker Result. ................................................................................ 9 Figure 2.5 Intuitive description of Mean-Shift.[14] ................................................... 10 Figure 2.6 Summary of CAMSHIFT algorithm. ........................................................ 13 Figure 2.7 LBP and CS-LBP features for a neighborhood of 8 pixels [16]................... 15 Figure 2.8 Example of LBP calculation[16]............................................................... 16 Figure 2.9 LBP Tracker Result in second video.......................................................... 16 Figure 2.10 The core classifier system: detector, recognizer and tracker.[20] ............ 18 Figure 2.11 Comparing LBP Image and its back projection image. ............................ 21 Figure 2.12 SURF and CAMSHIFT 1......................................................................... 23 Figure 2.13 SURF and CAMSHIFT 2. ....................................................................... 23 Figure 2.14 CAMSHIFT with new HSV model. ......................................................... 26 Figure 2.15 CAMSHIFT improvement with hue-distance saturation features. ........... 29 Figure 2.16 Foreground extraction. .......................................................................... 29 Figure 2.17 Sample of elongated object.....................................................................30 Figure 2.18 Background subtraction in static background. ....................................... 31 Figure 2.19 Background subtraction in dynamic background. ................................... 33 Figure 3.1 A sample of complex shape object ............................................................ 34 Figure 3.2 Object Localization using only region growing ......................................... 35 Figure 3.3 More precise object localization with only a single click. .......................... 35 Figure 3.4 Text file configuration to tune the parameters ......................................... 36 Figure 3.5 Color mask illustration ............................................................................ 38 Figure 3.6 Segmentation for smoothing and noise removal of third test video........... 38 Figure 3.7 Histogram Back Projection of first test video ........................................... 39 Figure 3.8 Maximum rectangle illustration. .............................................................40 Figure 3.9 The proposed methods schema .............................................................. 41 Figure 4.1 Hue histogram of air plane body. ............................................................. 43 Figure 5.1 First video result with the proposed method. ........................................... 44 Figure 5.2 First video result with classic CAMSHFT at frame 33. .............................. 45 Figure 5.3 Object localization comparison ................................................................ 45 Figure 5.4 Second video result with our proposed method. ....................................... 47 Figure 5.5 Second video result with classic CAMSHIT. ............................................. 47 Figure 5.6 Third video result with the proposed method........................................... 48 Figure 5.7 Third video best result with classic CAMSHIFT at frame 300. .................. 50 Figure 5.8 Object (marked with red rectangle) tracked by the proposed method. ...... 50 Figure 5.9 Forth video best result with classic CAMSHIFT at frame 57. .................... 50 Figure 5.10 Drifting tracker...................................................................................... 51 Figure 5.11 Multiple object tracking using our proposed method .............................. 51

iv

Object Tracking: State of The Art and CAMSHIFT Improvement Using Multi-dominant Color Tracking

1

Introduction

Object tracking has been one of the most emerging areas in computer vision. There are a lot of applications of object tracking. One of which would be tracking an object in a clickable hypervideo. Hypervideo is a displayed video stream that contains embedded user-clickable anchors[19]. In this application, user can interact with the video like interaction between user with a website. This enriches the interactivity of a video. There will be a lot of advantages with this capability. For example, user can monetize their videos by putting companys links inside the video and, in reverse way, companies now able to promote their product in videos. Another capability that would be interesting is object tracking in hypervideo. This means user can select any object in a video and track along the video sequence. For example, user has favorite football player in football match and want to track his movement along the match, then it would be possible with this capability. This also true if a user want to track his favorite racer in F1 videos, track his favorite movie stars in a movie cinema, etc. In this thesis, we try to improve an object tracking method that can be used in hypervideo. Some state of the art of object tracking methods are reviewed, experimented and closely observed. We then select one of object tracking state of the art methods and improve it.

1.1

The General Aim of The Master Thesis1) Study some state of the art object tracking methods 2) Choose one to improve based on some criteria 3) Improve the chosen method with some constraint if needed

The general objective of the master thesis can be summarized into these points:

1

Object Tracking: State of The Art and CAMSHIFT Improvement Using Multi-dominant Color Tracking

2

Previous Work

Object tracking is a very wide area in computer vision. There are many kinds of method which are sometimes suitable only for specific conditions. This part will describe the review of some state of the art object tracking methods available now.

2.1

Test Videos

Before we go deeper into the state of the art methods, in this section we present some test videos...