Benchmarking of NSF Unidata MetPy in a CI/CD Workflow

Jaye Norman
Jaye Norman

I had no idea what to expect coming into my summer 2025 internship with NSF Unidata. I was entering right as a furlough began for the majority of the staff, and I was warned that the internship might look a little different than most years. But I decided to still give everything I could to this internship, and in return I expanded my software development knowledge by orders of magnitude and set myself up for not only an amazing summer, but a successful career and future.

Benchmark results grid
airspeed velocity benchmark results grid (click to enlarge)

When presented with a list of possible internship project ideas, one that caught my eye was the benchmarking of MetPy using the open-source software airspeed velocity (asv). I had never worked in a Continuous Integration/Continuous Deployment (CI/CD) context before, and expanding my knowledge of this DevOps concept seemed intriguing. In a convenient coincidence, Linfeng Li, the other NSF Unidata summer intern, chose to work on adding C++ into MetPy to make it faster. This worked out perfectly; he implemented changes to make MetPy more efficient while I implemented changes to measure by exactly how much.

The first few weeks were spent installing and configuring asv while writing documentation about my steps. I got very acquainted with the terminal as well as the nuances of running software on different operating systems. I also began writing benchmarks for MetPy’s functions, using python code to write and execute these functions with an artificial 4D dataset.

Benchmarking datasets
Benchmarking dataset (click to enlarge)

After I’d gotten asv working locally, it was time to think about the CI/CD component of this project. We wanted the benchmarking to happen automatically – we shouldn’t have to lift a finger. This is where GitHub Actions, Jenkins, and Docker software came into play. I learned how to integrate each of these into my own repository’s workflow as a sandbox before adding them to MetPy. GitHub Actions is used to generate the webpages from the asv results, and an NSF Unidata Jenkins instance is used to run the benchmarks weekly in a Docker container. This ensures that the hardware for the benchmarks is consistent and we can safely compare results from different benchmark runs. You can view the benchmarking results for MetPy versions 1.4 and later in NSF Unidata's MetPy-benchmark repository.

As with anything worth doing, some days were challenging. I had never worked in a CI/CD workflow, never run a benchmark, and never even heard of Docker software before. I had a lot to learn in the 11 weeks I was here; naturally, there were bugs and bumps in the road. But through amazing mentorship, the help of staff across the NSF Unidata roster, and good old fashioned persistence, I was able to create an end result that I’m proud of and an experience that I will take with me for the rest of my career.

Comments:

Post a Comment:
  • HTML Syntax: Allowed
Unidata Developer's Blog
A weblog about software development by Unidata developers*
Unidata Developer's Blog
A weblog about software development by Unidata developers*

Welcome

FAQs

News@Unidata blog

Take a poll!

What if we had an ongoing user poll in here?

Browse By Topic
Browse by Topic
« August 2025
SunMonTueWedThuFriSat
     
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
      
Today