Numba bug hunting series: When 12 threads are slower than 1
I was benchmarking a change in the similarity search module of aeon, and one of my control measurements came out backwards. The baseline, the code that ships today, took 447 ms with one thread and 1157 ms with twelve threads. So it did not just fail to go faster. It became slower, and it did that every time I ran it.
aeon is a Python library for time series machine learning. Almost all of its
hot code is written with numba, which compiles
Python functions to machine code and can run loops on several threads with
prange. When you pass n_jobs=12 to a function, this is the number of numba
threads it will use.
In this post I go from that number to the single line of code that causes it. I also keep the three explanations that I believed for a while and had to drop.