Recovering 3D objects from 2D photos is an important application in the areas of computer vision, computer intelligence, feature recognition, and virtual reality. This paper describes an innovative and systematic method that integrates automatic feature extraction, automatic feature matching, manual revision, feature recovery, and model reconstruction into an effective and integrated 3D object recovery tool. The proposed method is a convenient and inexpensive way to recover 3D scenes and models directly from 2D photos. New automatic key-point selection and hierarchical matching algorithms were developed for matching 2D photos with wide baselines. The method uses a universal camera intrinsic matrix estimation technique to eliminate the need for camera calibration experiments. A new automatic texture-mapping algorithm was also developed for finding the best textures in 2D photos. The paper includes some examples and results to show the capabilities of the new method.

1.
Cornelis
,
K.
,
Pollefeys
,
M.
,
Vergauwen
,
M.
, and
Van Gool
,
L.
, 2000, “
Augmented Reality Using Uncalibrated Video Sequences
,”
2nd European Workshop on 3D Structure from Multiple Images of Large-Scale Environments (SMILE2000)
,
Dublin, Irleand
, pp.
144
160
.
2.
Pollefeys
,
M.
, 1999, “
Self-Calibration and Metric 3D Reconstruction From Uncalibrated Image Sequences
,” Ph.D. thesis, Katholieke Universiteit Leuven, Heverlee, Belgium.
3.
Reed
,
M.
, and
Allen
,
P.
, 1999, “
3-D Modeling From Range Imagery: An Incremental Method With a Planning Component
,”
Image Vis. Comput.
0262-8856,
17
, pp.
99
111
.
4.
Stamos
,
I.
, and
Allen
,
P.
, 2000, “
3-D Model Construction Using Range and Image Data
,”
Computer Vision & Pattern Recognition Conf. (CVPR)
, pp.
531
536
.
5.
Shum
,
H.
, and
Szeliski
,
R.
, 1999, “
Stereo Reconstruction From Multiperspective Panoramas
,”
7th International Conf. on Computer Vision (ICCV’99)
,
Kerkyra, Greece
, pp.
14
21
.
6.
Jebara
,
T.
,
Azarbayejani
,
A.
, and
Pentland
,
A.
, 1999, “
3D Structure From 2D Motion
,”
IEEE Signal Process. Mag.
1053-5888,
16
(
3
), pp.
66
84
.
7.
Baker
,
S.
,
Szeliski
,
R.
, and
Anandan
,
P.
, 1998, “
A Layered Approach to Stereo Reconstruction
,”
IEEE Computer Society Conf. on Computer Vision and Pattern Recognition (CVPR98)
, pp.
434
441
.
8.
Baillard
,
C.
, and
Zisserman
,
A.
, 2000, “
A Plane-Sweep Strategy for the 3rd Reconstruction of Buildings From Multiple Images
,”
Int. Arch. Photogramm. Remote Sens.
0924-2716,
32
(
2
), pp.
56
62
.
9.
Fitzgibbon
,
A. W.
, and
Cross
,
G.
, 1998, “
A. Zisserman, Automatic 3D Model Construction for Turn-Table Sequences
,”
Proc. European Workshop on 3D Structure from Multiple Images of Large-Scale Environments
, pp.
155
170
.
10.
Harris
,
C. G.
, and
Stephens
,
M. J.
, 1988, “
Combined Corner and Edge Detector
,”
Proc. 4th Alvey Vision Conf.
,
Manchester, England
, pp.
147
151
.
11.
Zhang
,
Z.
,
Deriche
,
R.
,
Faugeras
,
O.
, and
Luong
,
Q.
, 1995, “
A Robust Technique for Matching Two Uncalibrated Images Through the Recovery of the Unknown Epipolar Geometry
,”
Artif. Intell.
0004-3702,
78
, pp.
87
119
.
12.
Tissainayagam
,
P.
, and
Suter
,
D.
, 2004, “
Assessing the Performance of Corner Detectors for Point Feature Tracking Applications
,”
Image Vis. Comput.
0262-8856,
22
(
8
), pp.
663
679
.
13.
Canny
,
J. F.
, 1983, “
Finding Edges and Lines in Images
,” Masters thesis, AI Lab, MIT.
14.
Gonzalez
,
R.
, and
Woods
,
R.
, 2002,
Digital Image Processing
,
2nd ed.
,
Prentice Hall
, Englewood Cliffs, NJ.
15.
Gao
,
J.
,
Kosaka
,
A.
, and
Kak
,
A.
, 1998, “
A Deformable Model for Human Organ Extraction
,”
Proc. IEEE International Conf. on Image Processing
, Vol.
3
, pp.
323
327
.
16.
Pham
,
D. L.
,
Xu
,
C.
, and
Prince
,
J. L.
, 2000, “
A Survey of Current Methods in Medical Image Segmentation
,”
Annu. Rev. Biomed. Eng.
1523-9829,
2
, pp.
315
337
.
17.
Ma
,
Y.
,
Soatto
,
S.
,
Kosecka
,
J.
, and
Sastry
,
S.
, 2003,
An Invitation to 3-D Vision: From Images to Geometric Models
,
Springer-Verlag
, Berlin.
18.
Loaiza
,
H.
,
Triboulet
,
J.
, and
Lelandais
,
S.
, 2001, “
Matching Segments in Stereoscopic Vision
,”
IEEE Instrum. Meas. Mag.
1094-6969,
4
(
1
), pp.
37
42
.
19.
Deriche
,
R.
, and
Faugeras
,
O.
, 1990, “
2-D Curve Matching Using High Curvature Points: Application to Stereo Vision
,”
Proc. International Conf. on Pattern Recognition
. New Jersey, pp.
240
242
.
20.
Han
,
J. H.
, and
Park
,
J. S.
, 2000, “
Contour Matching Using Epipolar Geometry
,”
IEEE Trans. Pattern Anal. Mach. Intell.
0162-8828,
22
(
4
), pp.
358
370
.
21.
Tuytelaars
,
T.
,
Vergauwen
,
M.
,
Pollefeys
,
M.
, and
Van Gool
,
L.
, 1999, “
Image Matching for Wide Baseline Stereo
,”
Proc. International Conf. on Forensic Human Identification
.
22.
Xu
,
G.
,
Terai
,
J.
, and
Shum
,
H.
, 2000, “
A Linear Algorithm for Camera Self-Calibration, Motion and Structure Recovery for Multi-Planar Scenes From Two Perspective Images
,”
Computer Vision & Pattern Recognition Conf. (CVPR)
, pp.
2474
2479
.
23.
Vira
,
N.
, 2003, “
Modeling of a Three-dimensional Digital Image From 2D Stereo Paris
,”
Proceedings of the IASTED International Conference on Modeling and Simulation (MS 2003)
, pp.
482
488
.
24.
Zhang
,
Z.
, 1995, “
Estimating Motion and Structure From Correspondences of Line Segments Between Two Perspective Images
,”
IEEE Trans. Pattern Anal. Mach. Intell.
0162-8828,
17
(
12
), pp.
1129
1139
.
25.
Guermeur
,
P.
, and
Louchet
,
J.
, 2003, “
An Evolutionary Algorithm for Camera Calibration
,”
ICRODIC
,
Rethymnon, Greece
, pp.
799
804
.
26.
Debevec
,
P. E.
,
Taylor
,
C. J.
, and
Malik
,
J.
, 1996, “
Modeling and Rendering Architecture From Photographs
,” In
SIGGRAPH ’96
, August 1996, pp.
11
20
.
27.
Cipolla
,
R.
,
Robertson
,
D.
, and
Boyer
,
E.
, 1999, “
Photobuilder-3D Models of Architectural Scenes From Uncalibrated Images
,”
Proc. IEEE International Conference on Multimedia Computing and Systems
,
Firenze
, pp.
25
31
.
28.
Strecha
,
C.
,
Tuytelaars
,
T.
, and
Gool
,
L. V.
, 2003, “
Dense Matching of Multiple Wide-baseline Views
,”
ICCV 2003
, pp.
1194
1201
.
You do not currently have access to this content.