By Dave DeFusco
Researchers in the Katz School鈥檚 Department of Graduate Computer Science and Engineering have developed a new artificial intelligence method that could make advanced optimization techniques much faster and more accessible.
The paper, 鈥,鈥 introduces TuRBO-ENN, a system designed to speed up Bayesian optimization. This technique helps researchers find the best solution to a complex problem without testing every possibility. It is widely used in engineering simulations, robotics, scientific research and AI applications.
The study was led by Mehul Bafna, a graduate of the M.S. in Artificial Intelligence, and Siddhant Anand Jadhav, a student in the program, with Adjunct Professor David Sweet serving as corresponding author.
Traditional Bayesian optimization relies on a mathematical tool called a Gaussian process, or GP, to predict which solutions are most promising. While effective, these models become increasingly slow and computationally expensive as more data is collected.
鈥淭raditional Bayesian optimization methods are very good at predicting which solutions might work best,鈥 said Bafna, 鈥渂ut the mathematical system they rely on is extremely expensive to run and requires a lot of computing power.鈥
That challenge inspired the researchers to develop Epistemic Nearest Neighbors, or ENN, a simpler alternative. Instead of building a complex model from all previous observations, ENN focuses on nearby examples, much like a recommendation system identifies users with similar interests.
鈥淭raditional Gaussian process systems become dramatically slower as more data is added,鈥 said Bafna. 鈥淚f you want Bayesian optimization to handle huge numbers of fast, inexpensive simulations, you need a method that scales much more efficiently as the data grows.鈥
According to the study, TuRBO-ENN reduced proposal time鈥攖he time required to generate the next candidate solution鈥攂y as much as two orders of magnitude while maintaining similar optimization performance.
The research focuses on situations where simulations run very quickly, allowing tens of thousands of observations to be generated during a single optimization process. In these settings, the time spent making predictions can become a major bottleneck.
Unlike Gaussian-process systems, which slow significantly as datasets grow, ENN scales much more efficiently. The method also estimates uncertainty in its predictions and uses that information to guide its search for better solutions.
The researchers demonstrated that the simplified approach remained competitive with traditional methods while dramatically reducing computation time.
鈥淏ayesian optimization has historically been very powerful, but the computational costs can become overwhelming at scale,鈥 said Sweet. 鈥淭his research shows that you can simplify the surrogate model substantially while still preserving strong optimization performance.鈥
