Abstract

Deep neural networks (DNNs) have been successful in classification and retrieval tasks of images and text, as well as in the graphics domain. However, these DNNs algorithms do not translate to 3D engineering models used in the product design and manufacturing. This paper studies the use of multi-view convolutional neural network (MVCNN) algorithm enhanced by the addition of engineering metadata, for classification and retrieval of 3D computer-aided design (CAD) models. The proposed algorithm (MVCNN++) builds on the MVCNN algorithm with the addition of part dimension data, improving its efficacy for manufacturing part classification and yielding an improvement in classification accuracy of 5.8% over the original version. Unlike datasets used for 3D shape classification and retrieval in the computer graphics domain, engineering level description of 3D CAD models do not yield themselves to neat, distinct classes. Techniques such as relaxed-classification and prime angled cameras for capturing feature detail were used to address training data capture issues specific to 3D CAD models, along with the use of transfer learning to reduce training time. Our study has shown that DNNs can be used to search and discover relevant 3D engineering models in large public repositories, making 3D models accessible to the community.

References

1.
LeCun
,
Y.
,
Bottou
,
L.
,
Bengio
,
Y.
, and
Haffner
,
P.
,
1998
, “
Gradient-based Learning Applied to Document Recognition
,”
Proc. IEEE
,
86
(
11
), pp.
2278
2324
. 10.1109/5.726791
2.
Su
,
H.
,
Maji
,
S.
,
Kalogerakis
,
E.
, and
Learned-Miller
,
E.
,
2015
, “
Multi-view Convolutional Neural Networks for
3D
Shape Recognition
,”
Proceedings of the IEEE International Conference on Computer Vision
,
Santiago, Chile
,
Dec. 11–18
, pp.
945
953
.
3.
Deng
,
J.
,
Dong
,
W.
,
Socher
,
R.
,
Li
,
L.-J.
,
Li
,
K.
, and
Fei-Fei
,
L.
,
2009
, “
Imagenet: A Large-Scale Hierarchical Image Database
,”
2009 IEEE Conference on Computer Vision and Pattern Recognition
,
FL
,
June 20–25
,
IEEE
, pp.
248
255
.
4.
Bharadwaj
,
A.
,
Xu
,
Y.
,
Angrish
,
A.
,
Chen
,
Y.
, and
Starly
,
B.
,
2019
, “
Development of a Pilot Manufacturing Cyberinfrastructure With an Information Rich Mechanical CAD 3D Model Repository
,”
Proceedings of the ASME 2019 14th International Manufacturing Science and Engineering Conference. Volume 1: Additive Manufacturing; Manufacturing Equipment and Systems; Bio and Sustainable Manufacturing
,
Erie, PA
,
June 10–14
, ASME, p. V001T02A035
5.
Jayanti
,
S.
,
Kalyanaraman
,
Y.
,
Iyer
,
N.
, and
Ramani
,
K.
,
2006
, “
Developing an Engineering Shape Benchmark for CAD Models
,”
Comput.-Aided Des.
,
38
(
9
), pp.
939
953
. 10.1016/j.cad.2006.06.007
6.
Koch
,
S.
,
Matveev
,
A.
,
Jiang
,
Z.
,
Williams
,
F.
,
Artemov
,
A.
,
Burnaev
,
E.
,
Alexa
,
M.
,
Zorin
,
D.
, and
Panozzo
,
D.
,
2019
, “
ABC: A Big CAD Model Dataset For Geometric Deep Learning
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
,
Long Beach, FL
,
June 10–15
, pp.
9601
9611
.
7.
Sadjadi
,
F. A.
, and
Hall
,
E. L.
,
1980
, “
Three-dimensional Moment Invariants
,”
IEEE Trans. Pattern Anal. Mach. Intell.
,
2
(
2
), pp.
127
136
. 10.1109/TPAMI.1980.4766990
8.
Cybenko
,
G.
,
Bhasin
,
A.
, and
Cohen
,
K. D.
,
1997
, “
Pattern Recognition of 3D CAD Objects: Towards an Electronic Yellow Pages of Mechanical Parts
,”
Int. J. Smart Eng. Syst. Des.
,
1
(
1
), pp.
1
13
.
9.
Tangelder
,
J. W. H.
, and
Veltkamp
,
R. C.
,
2007
, “
A Survey of Content Based 3D Shape Retrieval Methods
,”
Multimed Tools Appl.
,
39
(
3
), p.
441
. https://dx.doi.org/10.1007/s11042-007-0181-0
10.
Saupe
,
D.
, and
Vranić
,
D. V.
,
2001
, “3D Model Retrieval With Spherical Harmonics and Moments,”
Joint Pattern Recognition Symposium
,
B.
Radig
and
S.
Florczyk
, eds., DAGM 2001. Lecture Notes in Computer Science, Vol.
2191
,
Springer
,
Berlin, Heidelberg
, pp.
392
397
.
11.
Kazhdan
,
M.
,
Funkhouser
,
T.
, and
Rusinkiewicz
,
S.
,
2003
, “
Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors
,”
Eurographics Symposium on Geometry Processing
,
June
,
L.
Kobbelt
,
P.
Schröder
, and
H.
Hoppe
, eds., Vol.
6
, pp.
156
164
.
12.
Angrish
,
A.
,
Craver
,
B.
, and
Starly
,
B.
,
2019
, “
“FabSearch”: A 3D CAD Model-Based Search Engine for Sourcing Manufacturing Services
,”
ASME J. Comput. Inf. Sci. Eng.
,
19
(
4
), p.
041006
. 10.1115/1.4043211
13.
Cicirello
,
V.
, and
Regli
,
W. C.
,
2001
, “
Machining Feature-Based Comparisons of Mechanical Parts
,”
Proceedings International Conference on Shape Modeling and Applications
, ACM SIGGRAPH, the Computer Graphics Society and EUROGRAPHICS, IEEE Computer, pp.
176
187
.
14.
Han
,
J.
,
Pratt
,
M.
, and
Regli
,
W. C.
,
2000
, “
Manufacturing Feature Recognition From Solid Models: A Status Report
,”
IEEE Trans. Rob. Autom.
,
16
(
6
), pp.
782
796
. 10.1109/70.897789
15.
El-Mehalawi
,
M.
, and
Allen Miller
,
R.
,
2003
, “
A Database System of Mechanical Components Based on Geometric and Topological Similarity. Part II: Indexing, Retrieval, Matching, and Similarity Assessment
,”
Comput.-Aided Des.
,
35
(
1
), pp.
95
105
. 10.1016/S0010-4485(01)00178-6
16.
Bespalov
,
D.
,
Regli
,
W. C.
, and
Shokoufandeh
,
A.
,
2008
, “
Reeb Graph Based Shape Retrieval for CAD
,”
ASME 2003 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
,
Chicago, IL
,
Sept. 2–6, 2003
,
American Society of Mechanical Engineers Digital Collection
, pp.
229
238
.
17.
Sundar
,
H.
,
Silver
,
D.
,
Gagvani
,
N.
, and
Dickinson
,
S.
,
2003
, “
Skeleton Based Shape Matching and Retrieval
,”
2003 Shape Modeling International
,
Seoul, South Korea
,
May 12–16
,
IEEE Computer Society
, pp.
130
139
.
18.
Shah
,
J. J.
, and
Bhatnagar
,
A. S.
,
1989
, “
Group Technology Classification From Feature-Based Geometric Models
,”
Manuf. Rev.
,
2
(
3
), pp.
204
213
.
19.
Chung
,
Y.
, and
Kusiak
,
A.
,
1994
, “
Grouping Parts With a Neural Network
,”
J. Manuf. Syst.
,
13
(
4
), pp.
262
275
. 10.1016/0278-6125(94)90034-5
20.
Iyer
,
N.
,
Jayanti
,
S.
,
Lou
,
K.
,
Kalyanaraman
,
Y.
, and
Ramani
,
K.
,
2005
, “
Three-dimensional Shape Searching: State-of-the-art Review and Future Trends
,”
Comput.-Aided Des.
,
37
(
5
), pp.
509
530
. 10.1016/j.cad.2004.07.002
21.
Cardone
,
A.
,
Gupta
,
S. K.
, and
Karnik
,
M.
,
2003
, “
A Survey of Shape Similarity Assessment Algorithms for Product Design and Manufacturing Applications
,”
ASME J. Comput. Inf. Sci. Eng.
,
3
(
2
), pp.
109
118
. 10.1115/1.1577356
22.
Prabhakar
,
S.
, and
Henderson
,
M. R.
,
1992
, “
Automatic Form-Feature Recognition Using Neural-Network-Based Techniques on Boundary Representations of Solid Models
,”
Comput.-Aided Des.
,
24
(
7
), pp.
381
393
. 10.1016/0010-4485(92)90064-H
23.
Kaparthi
,
S.
, and
Suresh
,
N. C.
,
1991
, “
A Neural Network System for Shape-Based Classification and Coding of Rotational Parts
,”
Int. J. Prod. Res.
,
29
(
9
), pp.
1771
1784
. 10.1080/00207549108948048
24.
Krizhevsky
,
A.
,
Sutskever
,
I.
, and
Hinton
,
G. E.
,
2017
, “Imagenet Classification With Deep Convolutional Neural Networks,”
Commun. ACM
,
60
, pp.
84
90
. https://doi.org/10.1145/3065386
25.
He
,
K.
,
Zhang
,
X.
,
Ren
,
S.
, and
Sun
,
J.
,
2016
, “
Deep Residual Learning for Image Recognition
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
,
Las Vegas, NV
,
June 26–July 1
, pp.
770
778
.
26.
Chang
,
A. X.
,
Funkhouser
,
T.
,
Guibas
,
L.
,
Hanrahan
,
P.
,
Huang
,
Q.
,
Li
,
Z.
,
Savarese
,
S.
et al
,
2015
, “
Shapenet: An information-rich 3d model repository
.” .
27.
Wu
,
Z.
,
Song
,
S.
,
Khosla
,
A.
,
Yu
,
F.
,
Zhang
,
L.
,
Tang
,
X.
, and
Xiao
,
J.
,
2015
, “
3D Shapenets: A Deep Representation for Volumetric Shapes
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
,
Boston, MA
,
June 8–10
, pp.
1912
1920
.
28.
Maturana
,
D.
, and
Scherer
,
S.
,
2015
, “
Voxnet: A 3D Convolutional Neural Network for Real-Time Object Recognition
,”
2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
,
Hamburg, Germany
,
Sept. 28–Oct. 2
, pp.
922
928
.
29.
Zhang
,
Z.
,
Jaiswal
,
P.
, and
Rai
,
R.
,
2018
, “
FeatureNet: Machining Feature Recognition Based on 3D Convolution Neural Network
,”
Comput.-Aided Des.
,
101
, pp.
12
22
. 10.1016/j.cad.2018.03.006
30.
Simonyan
,
K.
, and
Zisserman
,
A.
,
2014
, “
Very Deep Convolutional Networks for Large-scale Image Recognition
.” arXiv preprint arXiv:1409.1556.
31.
Pan
,
S. J.
, and
Yang
,
Q.
,
2010
, “
A Survey on Transfer Learning
,”
IEEE Trans. Knowl. and Data Eng.
,
22
(
10
), pp.
1345
1359
doi:10.1109/TKDE.2009.191.
32.
Pan
,
S. J.
, and
Yang
,
Q.
,
2009
, “
A Survey on Transfer Learning
,”
IEEE Trans. Knowl. Data Eng.
,
22
(
10
), pp.
1345
1359
. 10.1109/TKDE.2009.191
33.
Rozantsev
,
A.
,
Salzmann
,
M.
, and
Fua
,
P.
,
2018
, “
Beyond Sharing Weights for Deep Domain Adaptation
,”
IEEE Trans. Pattern Anal. Mach. Intell.
,
41
(
4
), pp.
801
814
. 10.1109/TPAMI.2018.2814042
34.
Zhang
,
Z.
,
Ning
,
G.
, and
He
,
Z.
,
2017
, “
Knowledge Projection for Deep Neural Networks
.” .
35.
Kanezaki
,
A.
,
Matsushita
,
Y.
, and
Nishida
,
Y.
,
2018
, “
Rotationnet: Joint Object Categorization and Pose Estimation Using Multiviews From Unsupervised Viewpoints
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
,
Salt Lake City, UT
,
June 18–23
, pp.
5010
5019
.
36.
Feng
,
Y.
,
Zhang
,
Z.
,
Zhao
,
X.
,
Ji
,
R.
, and
Gao
,
Y.
,
2018
, “
GVCNN: Group-View Convolutional Neural Networks for 3D Shape Recognition
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
, pp.
264
272
.
37.
Starly
,
B.
,
Akshay
,
B.
, and
Angrish
,
A.
,
FabWave CAD Repository Categorized Part Classes
. 10.13140/RG.2.2.31167.87201
38.
Keras
,
2020
, “
Keras: Deep Learning Library for Theano and Tensorflow
.” URL: https://github.com/keras-team/keras, no. 8:T1.
39.
Abadi
,
M.
,
Agarwal
,
A.
,
Barham
,
P.
,
Brevdo
,
E.
,
Chen
,
Z.
,
Citro
,
C.
,
Corrado
,
G. S.
et al
,
2016
, “Tensorflow: Large-scale Machine Learning on Heterogeneous Distributed Systems.” .
You do not currently have access to this content.