Abstract

Microbubble enhanced high intensity focused ultrasound (HIFU) is of great interest to tissue ablation for solid tumor treatments such as in liver and brain cancers, in which contrast agents/microbubbles are injected into the targeted region to promote heating and reduce prefocal tissue damage. A compressible Euler–Lagrange coupled model has been developed to accurately characterize the acoustic and thermal fields during this process. This employs a compressible Navier–Stokes solver for the ultrasound acoustic field and a discrete singularities model for bubble dynamics. To address the demanding computational cost relevant to practical medical applications, a multilevel hybrid message-passing interface (MPI)-open multiprocessing (OpenMP) parallelization scheme is developed to take advantage of both scalability of MPI and load balancing of OpenMP. At the first level, the Eulerian computational domain is divided into multiple subdomains and the bubbles are subdivided into groups based on which subdomain they fall into. At the next level, in each subdomain containing bubbles, multiple OpenMP threads are activated to speed up the computations of the bubble dynamics. For improved throughput, the OpenMP threads are more heavily distributed to subdomains where the bubbles are clustered. By doing this, MPI load imbalance issue due to uneven bubble distribution is mitigated by OpenMP speedup locally for those subdomains hosting more bubbles than others. The hybrid MPI-OpenMP Euler–Lagrange solver is used to conduct simulations and physical studies of bubble-enhanced HIFU problems containing a large number of microbubbles. The phenomenon of acoustic shadowing caused by the bubble cloud is then analyzed and discussed. Efficiency tests on two different machines with 48 processors are conducted and indicate 2–3 times speedup with the same hardware by introducing an OpenMP parallelization in combination with the MPI parallelization.

References

1.
Kennedy
,
J. E.
,
Wu
,
F.
,
ter Haar
,
G. R.
,
Gleeson
,
F. V.
,
Philips
,
R. P.
,
Middleton
,
M. R.
, and
Cranston
,
D.
,
2004
, “
High-Intensity Focused Ultrasound for the Treatment of Liver Tumours
,”
Ultrasonics
,
42
(
1–9
), pp.
931
935
.10.1016/j.ultras.2004.01.089
2.
Hynynen
,
K.
, and
Clement
,
G.
,
2009
, “
Clinical Applications of Focused Ultrasound—The Brain
,”
Int. J. Hyperthermia
,
23
(
2
), pp.
193
202
.10.1080/02656730701200094
3.
Kajiyama
,
K.
,
Yoshinaka
,
S.
,
Takagi
,
Y.
, and
Matsumoto
,
Y.
,
2010
, “
Micro-Bubble Enhanced HIFU
,”
Phys. Procedia
,
3
, pp.
305
314
.10.1016/j.phpro.2010.01.041
4.
Razansky
,
D.
,
Einziger
,
P. D.
, and
Adam
,
D. R.
,
2006
, “
Enhanced Heat Deposition Using Ultrasound Contrast Agent—Modeling and Experimental Observations
,”
IEEE Trans. Ultrason. Ferroelectr. Freq. Control
,
53
, pp.
137
147
.10.1109/TUFFC.2006.1588399
5.
Chung
,
D. J.
,
Cho
,
S. H.
,
Lee
,
J. M.
, and
Hahn
,
S. T.
,
2012
, “
Effect of Microbubble Contrast Agent During High Intensity Focused Ultrasound Ablation on Rabbit Liver In Vivo
,”
Eur. J. Radiol.
,
81
(
4
), pp.
e519
e523
.10.1016/j.ejrad.2011.06.002
6.
Kaneko
,
Y.
,
Maruyama
,
T.
,
Takegami
,
K.
,
Watanabe
,
T.
, and
Mitsui
,
H.
,
2005
, “
Use of a Microbubble Agent to Increase the Effects of High Intensity Focused Ultrasound on Liver Tissue
,”
Eur. Radiol.
,
15
, pp.
1415
1420
.10.1007/s00330-005-2663-7
7.
Gnanaskandan
,
A.
,
Hsiao
,
C.-T.
, and
Chahine
,
G. L.
,
2018
, “
Numerical Simulation of High Intensity Focused Ultrasound (HIFU) Using a Fully Compressible Multiscale Model
,”
Proceedings of the 10th International Symposium on Cavitation
(
CAV2018
), Baltimore, MD, May 14–16.10.1115/1.861851_ch150
8.
Gnanaskandan
,
A.
,
Hsiao
,
C.-T.
, and
Chahine
,
G. L.
,
2019
, “
Modeling of Microbubble-Enhanced High-Intensity Focused Ultrasound
,”
Ultrasound Med. Biol.
, 115, pp.
25
45
.10.1016/j.ultrasmedbio.2019.02.022
9.
Dinsmore
,
C.
,
Aminfar
,
A.
, and
Princevac
,
M.
,
2017
, “
Dissipative Effects of Bubbles and Particles in Shear Flows
,”
ASME J. Fluids Eng.
,
139
(
6
), p.
061302
.10.1115/1.4035946
10.
Kinzel
,
M. P.
,
Lindau
,
J. W.
, and
Kunz
,
R. F.
,
2019
, “
An Assessment of Computational Fluid Dynamics Cavitation Models Using Bubble Growth Theory and Bubble Transport Modeling
,”
ASME J. Fluids Eng.
,
141
(
4
), p.
041301
.10.1115/1.4042421
11.
Khojasteh-Manesh
,
M.
, and
Mahdi
,
M.
,
2019
, “
Evaluation of Cavitation Erosion Intensity in a Microscale Nozzle Using Eulerian–Lagrangian Bubble Dynamic Simulation
,”
ASME J. Fluids Eng.
,
141
(
6
), p.
061303
.10.1115/1.4042960
12.
Ma
,
J.
,
Gnanaskandan
,
A.
,
Hsiao
,
C. T.
, and
Chahine
,
G. L.
,
2021
, “
Message Passing Interface Parallelization for Two-Way Coupled Euler-Lagrange Simulation of Microbubble Enhanced HIFU
,”
ASME J. Fluids Eng.
, 143(8), p.
081105
.10.1115/1.4051148
13.
Ma
,
J.
,
Hsiao
,
C. T.
, and
Chahine
,
G. L.
,
2015
, “
Shared-Memory Parallelization for Two-Way Coupled Euler-Lagrange Modeling of Cavitating Bubbly Flows
,”
ASME J. Fluids Eng.
,
137
(
12
), p.
121106
.10.1115/1.4030919
14.
Pelanti
,
M.
, and
Shyue
,
K.-M.
,
2014
, “
A Mixture-Energy-Consistent Six-Equation Two-Phase Numerical Model for Fluids With Interfaces, Cavitation and Evaporation Waves
,”
J. Comput. Phys.
,
259
, pp.
331
357
.10.1016/j.jcp.2013.12.003
15.
Colella
,
P.
,
1985
, “
A Direct Eulerian MUSCL Scheme for Gas Dynamics
,”
SIAM J. Sci. Stat. Comput.
,
6
(
1
), pp.
104
117
.10.1137/0906009
16.
Kapahi
,
A.
,
Hsiao
,
C.
, and
Chahine
,
G.
,
2015
, “
A Multi-Material Flow Solver for High Speed Compressible Flows
,”
Comput. Fluids
, 2, pp.
163
179
.10.1016/j.compfluid.2015.03.016
17.
Plesset
,
M. S.
, and
Prosperetti
,
A.
,
1977
, “
Bubble Dynamics and Cavitation
,”
Annu. Rev. Fluid Mech.
,
9
, pp.
145
185
.10.1146/annurev.fl.09.010177.001045
18.
Johnson
,
V. E.
, and
Hsieh
,
T.
,
1966
, “
The Influence of the Trajectories of Gas Nuclei on Cavitation Inception
,”
Sixth Symposium on Naval Hydrodynamics
, Washington, DC, Sept. 27, pp.
163
179
.
19.
Ma
,
J.
,
Hsiao
,
C.-T.
, and
Chahine
,
G. L.
,
2015
, “
Spherical Bubble Dynamics in a Bubbly Medium Using an Euler–Lagrange Model
,”
Chem. Eng. Sci.
,
128
, pp.
64
81
.10.1016/j.ces.2015.01.056
20.
Ma
,
J.
,
Hsiao
,
C.-T.
, and
Chahine
,
G. L.
,
2015
, “
Euler-Lagrange Simulations of Bubble Cloud Dynamics Near a Wall
,”
ASME J. Fluids Eng.
,
137
(
4
), pp.
041301
041310
.10.1115/1.4028853
21.
Ma
,
J.
,
Hsiao
,
C.-T.
, and
Chahine
,
G. L.
, January
2018
, “
Numerical Study of Acoustically Driven Bubble Cloud Dynamics Near a Rigid Wall
,”
Ultrason. Sonochem.
,
40
, pp.
944
954
.10.1016/j.ultsonch.2017.08.033
22.
Okita
,
K.
,
Sugiyama
,
K.
,
Takagi
,
S.
, and
Matsumto
,
Y.
,
2013
, “
Microbubble Behavior in an Ultrasound Field for High Intensity Focused Ultrasound Therapy Enhancement
,”
J. Acoust. Soc. Am.
,
134
, pp.
1576
1585
.10.1121/1.4812880
You do not currently have access to this content.