My summer internship at NSF Unidata has been a fulfilling and unforgettable experience. I arrived at the UCAR Foothills Lab in beautiful and cozy Boulder with a clear goal: to practice my C++ programming skills and have fun with any interesting software that could benefit my work on planetary numerical modeling. Despite the fact that most of the NSF Unidata staff were on furlough when I arrived in mid-May, impacting the available projects, my mentors, NSF Unidata developers Drew Camron and Ryan May, ensured I could still settle down on some interesting projects.
Once the furlough ended and things returned to normal, the office was once again a warm and welcoming place. During the initial three weeks, I had the chance to explore several exciting projects. I worked with NSF Unidata developer Thomas Martin to delve into machine learning with planetary general circulation model outputs, and even though my focus ultimately shifted more toward physics-based problems, I gained a valuable understanding of how machine learning models extract key information from large datasets.
For the majority of the summer, I dedicated my time to two primary projects: accelerating the thermodynamics module of MetPy by swapping its Python core with C++, and creating a 3D visualization of Jupiter’s mid-latitude jet streams using NSF Unidata’s Integrated Data Viewer (IDV).
C++ Acceleration of MetPy Thermodynamics Module
I never imagined that my first major C++ project would involve developing for both C++ and Python simultaneously. Improving the performance of MetPy, an open-source Python package for meteorology, gave me a fantastic opportunity to learn how a Python program can be powered by a C++ computational core. With a primary focus on optimizing the calculation of CAPE (Convective Available Potential Energy), I converted many thermodynamics functions to C++ and used the pybind11 library to create a seamless bridge back to Python.
This process required careful architectural decisions to determine which logic should be migrated to C++ for performance and which should remain in Python for flexibility. I also learned how to vectorize a scalar C++ function to handle multi-dimensional array inputs efficiently. With significant help and guidance from my colleague Jaye Norman, we successfully benchmarked the new implementation. The results from airspeed velocity (asv), a Python benchmarking tool, demonstrate significant speedups, including a >50% runtime reduction for parcel rising profile calculations, a 20-50% reduction for determining key convective levels, and a final 6% reduction for the full CAPE calculation.
Jupiter 3D Visualization in Unidata's Integrated Data Viewer
My second project was an exciting exploration into how well the IDV, a tool designed for visualizing Earth data, could adapt to non-Earth datasets. A significant part involved creating and modifying the NCML file for the raw Jupiter simulation data, adjusting variable conventions, dimensions, and vertical scales to make the data recognizable to the software.
A major technical challenge was handling a 300 GB dataset on my 16 GB memory laptop. To manage memory consumption, I pre-computed key variables—like relative vorticity and cloud densities—and read the data into the IDV in smaller chunks. Ultimately, the data was successfully visualized on a 3D globe, resulting in a fascinating video that showcases the intricate vortices and the distinct band-and-zone structure of Jupiter's atmosphere.
Beyond the Internship: A New Scientific Application
As the summer approached its end, it suddenly occurred to me that my two projects could be combined. Why not use the newly accelerated MetPy to calculate CAPE for the Jupiter simulation data? This idea opens up two exciting avenues. First, it allows us to test how efficiently the new MetPy handles a massive, high-resolution (170x300x400) dataset. Second, it pushes us to explore what adaptations are needed for MetPy to process data from other planets, including consideration for different planetary sizes and rotation rates, planet size, different formulas for broader temperature and pressure ranges, and multiple condensable species.
CAPE is a critical metric for cloud formation and storm frequency. On Jupiter, storms are thought to govern the global distribution of key tracer species like ammonia vapor, and possibly water vapor. By using CAPE as a proxy for storm intensity, I can examine the consistency between storm potential in the simulation and the real-world lightning distributions observed by the microwave radiometer on NASA's Juno mission. This is a thrilling scientific question that I hope to explore further.
I have truly enjoyed my experience here at Unidata—learning new skills, tackling interesting challenges, and being prepared (or not) to be inspired by the new ideas of my colleagues.
Jupiter data courtesy of Huazhi Ge, California Institute of Technology, Large-Scale Atmospheric Dynamics in Jupiter’s Mid-latitude — Compositional Structure, Thermal Structure, Large-Scale Dynamics, and Forward-Modelling Compared to Radio Observation — Simulated by the Nonhydrostatic Model. https://agu.confex.com/agu/fm23/meetingapp.cgi/Paper/1436567
