Abstract

Many design problems involve reasoning about points in high-dimensional space. A common strategy is to first embed these high-dimensional points into a low-dimensional latent space. We propose that a good embedding should be isometric—i.e., preserving the geodesic distance between points on the data manifold in the latent space. However, enforcing isometry is non-trivial for common neural embedding models such as autoencoders. Moreover, while theoretically appealing, it is unclear to what extent is enforcing isometry necessary for a given design analysis. This paper answers these questions by constructing an isometric embedding via an isometric autoencoder, which we employ to analyze an inverse airfoil design problem. Specifically, the paper describes how to train an isometric autoencoder and demonstrates its usefulness compared to non-isometric autoencoders on the UIUC airfoil dataset. Our ablation study illustrates that enforcing isometry is necessary for accurately discovering clusters through the latent space. We also show how isometric autoencoders can uncover pathologies in typical gradient-based shape optimization solvers through an analysis on the SU2-optimized airfoil dataset, wherein we find an over-reliance of the gradient solver on the angle of attack. Overall, this paper motivates the use of isometry constraints in neural embedding models, particularly in cases where researchers or designers intend to use distance-based analysis measures to analyze designs within the latent space. While this work focuses on airfoil design as an illustrative example, it applies to any domain where analyzing isometric design or data embeddings would be useful.

References

1.
Panchal
,
J. H.
,
Fuge
,
M.
,
Liu
,
Y.
,
Missoum
,
S.
, and
Tucker
,
C.
,
2019
, “
Machine Learning for Engineering Design
,”
ASME J. Mech. Des.
,
141
(
11
), p.
110301
.
2.
Missoum
,
S.
, and
Tucker
,
C.
,
2019
, “
Topology Design With Conditional Generative Adversarial Networks
,”
Volume 2A: 45th Design Automation Conference of International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
,
Anaheim, CA
,
Aug. 18–21
.
3.
Yu
,
Y.
,
Hur
,
T.
,
Jung
,
J.
, and
Jang
,
I. G.
,
2019
, “
Deep Learning for Determining a Near-Optimal Topological Design Without Any Iteration
,”
Struct. Multidiscipl. Optim.
,
59
(
3
), pp.
787
799
.
4.
Yilmaz
,
E.
, and
German
,
B.
,
2020
, “
Conditional Generative Adversarial Network Framework for Airfoil Inverse Design
,”
AIAA AVIATION 2020 FORUM
,
Virtual
,
June 15–19
, American Institute of Aeronautics and Astronautics.
5.
Chen
,
Q.
,
Wang
,
J.
,
Pope
,
P.
,
Chen
,
W.
, and
Fuge
,
M.
,
2022
, “
Inverse Design of Two-Dimensional Airfoils Using Conditional Generative Models and Surrogate Log-Likelihoods
,”
ASME J. Mech. Des.
,
144
(
2
), p.
021712
.
6.
Wang
,
J.
,
Chen
,
W. W.
,
Da
,
D.
,
Fuge
,
M.
, and
Rai
,
R.
,
2022
, “
IH-GAN: A Conditional Generative Model for Implicit Surface-Based Inverse Design of Cellular Structures
,”
Comput. Meth. Appl. Mech. Eng.
,
396
, p.
115060
.
7.
Nobari
,
A. H.
,
Chen
,
W.
, and
Ahmed
,
F.
,
2021
, “
Range-Gan: Range-Constrained Generative Adversarial Network for Conditioned Design Synthesis
.
8.
Yang
,
Z.
,
Li
,
X.
,
Catherine Brinson
,
L.
,
Choudhary
,
A. N.
,
Chen
,
W.
, and
Agrawal
,
A.
,
2018
, “
Microstructural Materials Design Via Deep Adversarial Learning Methodology
,”
ASME J. Mech. Des.
,
140
(
11
), p.
111416
.
9.
Mosser
,
L.
,
Dubrule
,
O.
, and
Blunt
,
M. J.
,
2017
, “
Reconstruction of Three-Dimensional Porous Media Using Generative Adversarial Neural Networks
,”
Phys. Rev. E
,
96
, p.
043309
.
10.
Xue
,
T.
,
Wallin
,
T. J.
,
Menguc
,
Y.
,
Adriaenssens
,
S.
, and
Chiaramonte
,
M.
,
2020
, “
Machine Learning Generative Models for Automatic Design of Multi-Material 3D Printed Composite Solids
,”
Ext. Mech. Lett.
,
41
, p.
100992
.
11.
Chen
,
W.
,
Chiu
,
K.
, and
Fuge
,
M.
,
2019
, “
Aerodynamic Design Optimization and Shape Exploration Using Generative Adversarial Networks
,”
AIAA Scitech 2019 Forum
,
San Diego, CA
,
Jan. 7–11
, p.
2351
.
12.
Chen
,
W.
, and
Ahmed
,
F.
,
2021
, “
PaDGAN: Learning to Generate High-Quality Novel Designs
,”
ASME J. Mech. Des.
,
143
(
3
), p.
031703
.
13.
Liang
,
L.
,
Liu
,
M.
,
Martin
,
C.
, and
Sun
,
W.
,
2018
, “
A Deep Learning Approach to Estimate Stress Distribution: A Fast and Accurate Surrogate of Finite-Element Analysis
,”
J. R. Soc. Interface.
,
15
(
138
), p.
20170844
.
14.
Jiang
,
H.
,
Nie
,
Z.
,
Yeo
,
R.
,
Farimani
,
A. B.
, and
Kara
,
L. B.
,
2021
, “
StressGAN: A Generative Deep Learning Model for Two-Dimensional Stress Distribution Prediction
,”
ASME J. Appl. Mech.
,
88
(
5
), p.
051005
.
15.
Nie
,
Z.
,
Jiang
,
H.
, and
Kara
,
L. B.
,
2019
, “
Stress Field Prediction in Cantilevered Structures Using Convolutional Neural Networks
,”
ASME J. Comput. Inf. Sci. Eng.
,
20
(
1
), p.
011002
.
16.
Kochkov
,
D.
,
Smith
,
J. A.
,
Alieva
,
A.
,
Wang
,
Q.
,
Brenner
,
M. P.
, and
Hoyer
,
S.
,
2021
, “
Machine Learning–Accelerated Computational Fluid Dynamics
,”
Proc. Natl. Acad. Sci. U. S. A.
,
118
(
21
), p.
e2101784118
.
17.
Kim
,
B.
,
Azevedo
,
V. C.
,
Thuerey
,
N.
,
Kim
,
T.
,
Gross
,
M.
, and
Solenthaler
,
B.
,
2019
, “
Deep Fluids: A Generative Network for Parameterized Fluid Simulations
,”
Comput. Graph. Forum
,
38
(
2
), pp.
59
70
.
18.
Bellman
,
R.
,
2010
,
Dynamic Programming
,
Princeton University Press
,
Princeton, NJ
.
19.
Belkin
,
M.
, and
Niyogi
,
P.
,
2001
, “Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering,”
Advances in Neural Information Processing Systems
, Vol.
14
,
T.
Dietterich
,
S.
Becker
, and
Z.
Ghahramani
, eds.,
MIT Press
,
Cambridge, MA
.
20.
Van der Maaten
,
L.
, and
Hinton
,
G.
,
2008
, “
Visualizing Data Using t-SNE
,”
J. Mach. Learn. Res.
,
9
(
11
), pp.
2579
2605
.
21.
McInnes
,
L.
,
Healy
,
J.
, and
Melville
,
J.
,
2018
, “
UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
,”
arXiv
. https://arxiv.org/abs/2109.02508
22.
Chen
,
W.
,
Fuge
,
M.
, and
Chazan
,
J.
,
2017
, “
Design Manifolds Capture the Intrinsic Complexity and Dimension of Design Spaces
,”
ASME J. Mech. Des.
,
139
(
5
), p.
051102
.
23.
Chen
,
N.
,
Klushyn
,
A.
,
Ferroni
,
F.
,
Bayer
,
J.
, and
Van Der Smagt
,
P.
,
2020
, “
Learning Flat Latent Manifolds With Vaes
,”
Proceedings of the 37th International Conference on Machine Learning
,
Virtual
,
July 13–18
.
24.
Gropp
,
A.
,
Atzmon
,
M.
, and
Lipman
,
Y.
,
2020
, “
Isometric Autoencoders
,”
arXiv
. https://arxiv.org/abs/2006.09289
25.
Yonghyeon
,
L.
,
Yoon
,
S.
,
Son
,
M.
, and
Park
,
F. C.
,
2022
, “
Regularized Autoencoders for Isometric Representation Learning
,”
International Conference on Learning Representations
,
Virtual
,
Apr. 22–25
.
26.
Ester
,
M.
,
Kriegel
,
H.-P.
,
Sander
,
J.
, and
Xu
,
X.
,
1996
, “
A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases With Noise
,”
KDD
,
Vol. 96
,
226
231
.
27.
Campello
,
R. J.
,
Kröger
,
P.
,
Sander
,
J.
, and
Zimek
,
A.
,
2020
, “
Density-Based Clustering
,”
Wiley Interdiscip. Rev.: Data Min. Knowl. Discov.
,
10
(
2
), p.
e1343
.
28.
SIGKDD, A.
,
2014
, “
Sigkdd Test of Time Award
.”
29.
Schubert
,
E.
,
Sander
,
J.
,
Ester
,
M.
,
Kriegel
,
H. P.
, and
Xu
,
X.
,
2017
, “
Dbscan Revisited, Revisited: Why and How You Should (Still) Use DBSCAN
,”
ACM Trans. Database Syst. (TODS)
,
42
(
3
), pp.
1
21
.
30.
Gan
,
J.
, and
Tao
,
Y.
,
2015
, “
Dbscan Revisited: Mis-Claim, Un-Fixability, and Approximation
,”
Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data
,
Melbourne, VIC, Australia
,
May 31– June 4
, pp.
519
530
.
31.
Ankerst
,
M.
,
Breunig
,
M. M.
,
Kriegel
,
H.-P.
, and
Sander
,
J.
,
1999
, “
Optics: Ordering Points to Identify the Clustering Structure
,”
ACM Sigmod Rec.
,
28
(
2
), pp.
49
60
.
32.
Hinneburg
,
A.
, and
Keim
,
D. A.
,
1998
,
An Efficient Approach to Clustering in Large Multimedia Databases With Noise
, Vol.
98
,
Bibliothek der Universität Konstanz
,
Konstanz, Germany
.
33.
McInnes
,
L.
,
Healy
,
J.
, and
Astels
,
S.
,
2017
, “
HDBSCAN: Hierarchical Density Based Clustering
,”
J. Open Source Softw.
,
2
(
11
), p.
205
.
34.
Johnson
,
S. C.
,
1967
, “
Hierarchical Clustering Schemes
,”
Psychometrika
,
32
(
3
), pp.
241
254
.
35.
MacQueen
,
J.
,
1967
, “
Classification and Analysis of Multivariate Observations
,”
Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability Held at the Statistical Laboratory University of California, Los Angeles, CA
, pp.
281
297
.
36.
Bishop
,
C. M.
, and
Nasrabadi
,
N. M.
,
2006
,
Pattern Recognition and Machine Learning
, Vol.
4
,
Springer
,
New York
.
37.
Andoni
,
A.
,
2009
, “
Nearest Neighbor Search: the Old, the New, and the Impossible
,”
Ph.D. Thesis
,
Massachusetts Institute of Technology
,
Cambridge, MA
,
38.
Kulesza
,
A.
, and
Taskar
,
B.
,
2012
, “
Determinantal Point Processes for Machine Learning
,”
Found. Trends. Mach. Learn.
,
5
(
2–3
), pp.
123
286
.
39.
Ahmed
,
F.
, and
Fuge
,
M.
,
2018
, “
Ranking Ideas for Diversity and Quality
,”
ASME J. Mech. Des.
,
140
(
1
), p.
011101
.
40.
Beyer
,
K.
,
Goldstein
,
J.
,
Ramakrishnan
,
R.
, and
Shaft
,
U.
,
1999
, “
When is ‘Nearest Neighbor’ Meaningful?
,”
Database Theory–ICDT’99: 7th International Conference, Proceedings 7, Jerusalem, Israel, Jan. 10–12
,
Springer
, pp.
217
235
.
41.
Aggarwal
,
C. C.
,
Hinneburg
,
A.
, and
Keim
,
D. A.
,
2001
, “
On the Surprising Behavior of Distance Metrics in High Dimensional Space
,”
Database Theory–ICDT 2001: 8th International Conference, Proceedings 8
,
London, UK
,
Jan. 4–6
,
Springer
, pp.
420
434
.
42.
Weinberger
,
K. Q.
, and
Saul
,
L. K.
,
2009
, “
Distance Metric Learning for Large Margin Nearest Neighbor Classification
,”
J. Mach. Learn. Res.
,
10
(
2
), pp.
207
244
.
43.
Fefferman
,
C.
,
Mitter
,
S.
, and
Narayanan
,
H.
,
2016
, “
Testing the Manifold Hypothesis
,”
J. Am. Math. Soc.
,
29
(
4
), pp.
983
1049
.
44.
Roth
,
P. M.
,
Hirzer
,
M.
,
Köstinger
,
M.
,
Beleznai
,
C.
, and
Bischof
,
H.
,
2014
, “Mahalanobis Distance Learning for Person Re-Identification,”
Person Re-Identification
,
Springer, London
,
UK
, pp.
247
267
.
45.
Lee
,
K.
,
Lee
,
K.
,
Lee
,
H.
, and
Shin
,
J.
,
2018
, “
A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
,”
Adv. Neural Inf. Process. Syst.
,
31
, pp.
7167
7177
.
46.
Bronstein
,
M. M.
,
Bruna
,
J.
,
Cohen
,
T.
, and
Veličković
,
P.
,
2021
, “
Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
,”
arXiv
. https://arxiv.org/abs/2104.13478
47.
Lee
,
J. M.
,
2006
,
Riemannian Manifolds: An Introduction to Curvature
, Vol.
176
,
Springer Science & Business Media
,
New York, NY
.
48.
Fletcher
,
P. T.
,
Lu
,
C.
,
Pizer
,
S. M.
, and
Joshi
,
S.
,
2004
, “
Principal Geodesic Analysis for the Study of Nonlinear Statistics of Shape
,”
IEEE Trans. Med. Imag.
,
23
(
8
), pp.
995
1005
.
49.
Nilsson
,
J.
,
Fioretos
,
T.
,
Höglund
,
M.
, and
Fontes
,
M.
,
2004
, “
Approximate Geodesic Distances Reveal Biologically Relevant Structures in Microarray Data
,”
Bioinformatics
,
20
(
6
), pp.
874
880
.
50.
Srivastava
,
A.
,
Klassen
,
E.
,
Joshi
,
S. H.
, and
Jermyn
,
I. H.
,
2010
, “
Shape Analysis of Elastic Curves in Euclidean Spaces
,”
IEEE Trans. Pattern Anal. Mach. Intell.
,
33
(
7
), pp.
1415
1428
.
51.
Tosi
,
A.
,
Hauberg
,
S.
,
Vellido
,
A.
, and
Lawrence
,
N. D.
,
2014
, “
Metrics for Probabilistic Geometries
,”
Proceedings of the Thirtieth Conference on Uncertainty in Artificial Intelligence
,
Quebec City, Quebec, Canada
,
July 23–27
.
52.
Chen
,
N.
,
Klushyn
,
A.
,
Kurle
,
R.
,
Jiang
,
X.
,
Bayer
,
J.
, and
Smagt
,
P.
,
2018
, “
Metrics for Deep Generative Models
,”
International Conference on Artificial Intelligence and Statistics
,
Playa Blanca, Lanzarote, Canary Islands
,
Apr. 9–11
, PMLR, pp.
1540
1550
.
53.
Linderman
,
G. C.
, and
Steinerberger
,
S.
,
2019
, “
Clustering With t-SNE, Provably
,”
SIAM J. Math. Data Sci.
,
1
(
2
), pp.
313
332
.
54.
Tenenbaum
,
J. B.
,
Silva
,
V. d.
, and
Langford
,
J. C.
,
2000
, “
A Global Geometric Framework for Nonlinear Dimensionality Reduction
,”
Science
,
290
(
5500
), pp.
2319
2323
.
55.
Roweis
,
S. T.
, and
Saul
,
L. K.
,
2000
, “
Nonlinear Dimensionality Reduction by Locally Linear Embedding
,”
Science
,
290
(
5500
), pp.
2323
2326
.
56.
Goodfellow
,
I.
,
Bengio
,
Y.
, and
Courville
,
A.
,
2016
,
Deep Learning
,
MIT Press
,
Cambridge, MA
.
57.
Coifman
,
R. R.
, and
Lafon
,
S.
,
2006
, “
Diffusion Maps
,”
Appl. Comput. Harmon. Anal.
,
21
(
1
), pp.
5
30
.
58.
Lee
,
J.
,
2010
,
Introduction to Topological Manifolds
, Vol.
202
,
Springer Science & Business Media
,
New York
.
59.
Asinomàs. Composition Equals Identity Function
. Mathematics Stack Exchange.
60.
Dinh
,
L.
,
Krueger
,
D.
, and
Bengio
,
Y.
,
2015
, “
Nice: Non-Linear Independent Components Estimation
,”
3rd International Conference on Learning Representations
,
San Diego, CA
,
May 7–9
.
61.
Dinh
,
L.
,
Sohl-Dickstein
,
J.
, and
Bengio
,
S.
,
2016
, “
Density Estimation Using Real NVP
,”
5th International Conference on Learning Representations
,
Toulon, France
,
Apr. 24–26
.
62.
Kingma
,
D. P.
, and
Dhariwal
,
P.
,
2018
, “
Glow: Generative Flow With Invertible 1 × 1 Convolutions
,”
Adv. Neural Inf. Process. Syst.
,
31
, pp.
10215
10224
.
63.
Zhang
,
H.
,
Cisse
,
M.
,
Dauphin
,
Y. N.
, and
Lopez-Paz
,
D.
,
2018
, “
Mixup: Beyond Empirical Risk Minimization
,”
6th International Conference on Learning Representations
,
Vancouver, BC, Canada
,
Apr. 30– May 3
.
64.
Zhu
,
J.-Y.
,
Park
,
T.
,
Isola
,
P.
, and
Efros
,
A. A.
,
2017
, “
Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks
,”
Proceedings of the IEEE International Conference on Computer Vision
,
Venice, Italy
,
Oct. 22–29
.
65.
Lee
,
J. M.
, and
Lee
,
J. M.
,
2012
,
Smooth Manifolds
,
Springer
,
New York, NY
.
66.
Levina
,
E.
, and
Bickel
,
P.
,
2004
, “
Maximum Likelihood Estimation of Intrinsic Dimension
,”
Adv. Neural Inf. Process. Syst.
,
17
, pp.
777
784
.
67.
MacKay
,
D. J.
, and
Ghahramani
,
Z.
,
2005
, “
Comments on ‘Maximum Likelihood Estimation of Intrinsic Dimension
” by E. Levina and P. Bickel.
68.
Chen
,
Q.
,
Pope
,
P.
, and
Fuge
,
M.
,
2022
, “
Learning Airfoil Manifolds With Optimal Transport
,”
AIAA SCITECH 2022 Forum
,
San Diego, CA
,
Jan. 3–7
, p.
2352
.
69.
Feydy
,
J.
,
Séjourné
,
T.
,
Vialard
,
F.-X.
,
Amari
,
S.-i.
,
Trouvé
,
A.
, and
Peyré
,
G.
,
2019
, “
Interpolating Between Optimal Transport and Mmd Using Sinkhorn Divergences
,”
The 22nd International Conference on Artificial Intelligence and Statistics
,
Naha, Okinawa, Japan
,
Apr. 16 –18
, PMLR, pp.
2681
2690
.
70.
Zhang
,
R.
,
Isola
,
P.
,
Efros
,
A. A.
,
Shechtman
,
E.
, and
Wang
,
O.
,
2018
, “
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
,
Salt Lake City, UT
,
June 18–22
, pp.
586
595
.
You do not currently have access to this content.