Computational Problem-Solving Tools

In Operations Research, solver choice is critical. While commercial options like CPLEX and Gurobi often dominate, there’s a strong ecosystem of open and freely available solvers worth knowing. The COIN-OR suite offers solid options like Cbc for MILP, Clp for LP, and Ipopt for nonlinear problems. Google OR-Tools is excellent for combinatorial optimization, routing, and CP/SAT, and includes its own LP/MIP solvers such as GLOP. GLPK, one of the most established open-source solvers, remains a go-to for LP and MIP—particularly in teaching and prototyping—though it can struggle with very large or complex problems. For quadratic programs, OSQP is a fast and reliable option, while OjAlgo provides a Java-based library for LP, QP, and MIP. Modeling frameworks like Pyomo and PuLP make it easy to define models and switch between solvers. While open-source solvers may not always match the performance of commercial ones on very large instances, they continue to advance rapidly and are invaluable for research, prototyping, and even production workflows. Which solvers do you typically use in your work? I'd love to hear what’s been working well for others.

🚀 Introducing Ultra-Fast Meta-Solvers for Solving PDEs! 🚀 Solving Partial Differential Equations (PDEs) just got smarter, faster, and more efficient! The paper "Automatic Discovery of Optimal Meta-Solvers via Multi-Objective Optimization" by Youngkyu Lee, Shanqing Liu, Jérôme Darbon, and George Em Karniadakis explores groundbreaking innovations in computational science. Here's what makes this work a game-changer: Highlights 🔧 Hybrid Meta-Solvers: Combines neural operators (like DeepONet) with classical iterative solvers (e.g., Jacobi, Gauss-Seidel) and Krylov methods (GMRES, BiCGStab). Neural networks serve as coarse preconditioners, tackling low-frequency errors, while iterative solvers handle high-frequency components. 📊 Multi-Objective Optimization: Automatically discovers the best solver by balancing performance metrics like speed, accuracy, and memory usage using Pareto optimality. 🎯 Preference-Based Solver Selection: Tailor solver choices to specific needs through user-defined preferences, ensuring optimal results for various applications. 💡 Scalable Parameterization: Meta-solvers are parameterized across neural operators, iterative methods, and multi-grid techniques to suit different problem domains. 🔍 Numerical Validation: Extensive experiments on 1D, 2D, and 3D Poisson equations reveal the best-performing solvers, showcasing efficiency improvements in diverse scenarios. 🔄 Extension to Nonlinear Systems: The methodology isn't just for linear problems—it holds promise for tackling nonlinear and time-dependent PDEs too! Applications 🌐 Uncertainty Quantification: Solve PDEs efficiently across varying conditions. 🏭 Large-Scale Simulations: Reduce computational time and memory in industrial and scientific problems. 🌊 Fluid Mechanics, Material Science, and Beyond: Push the boundaries of SciML applications. 📄 Paper Details Title: Automatic Discovery of Optimal Meta-Solvers via Multi-Objective Optimization Authors: Youngkyu Lee, Shanqing Liu, Jérôme Darbon, George Em Karniadakis Published: December 2024, arXiv preprint This research redefines computational efficiency, merging neural networks with classical solvers to achieve unmatched performance. A must-read for anyone in scientific machine learning (SciML), computational physics, or applied mathematics! 🔗 Read more and join the discussion: https://lnkd.in/d4C2hN-C #MachineLearning #PDEs #ScientificComputing #NeuralNetworks #Optimization #ResearchInnovation

Google's recent Gemini 2.5 report mentioned an fascinating advancement called "Deep Think" - a novel reasoning approach that enables AI models to generate multiple hypotheses in parallel and critically evaluate them before arriving at final answers. The results speak for themselves: state-of-the-art performance on challenging benchmarks including Olympiad mathematics, competitive coding, and multimodal reasoning tasks. What caught my attention was how this structured Chain-of-Thought approach could democratize advanced reasoning capabilities beyond proprietary models. So we built something similar. We developed an open-source DeepThink plugin for OptiLLM that brings these same parallel thinking techniques to open models like DeepSeek R1 and Qwen3. The plugin enables models to explore multiple solution paths simultaneously, evaluate different approaches, and converge on better answers through deeper reasoning processes. The technical implementation focuses on enhancing the reasoning pipeline during response generation, giving models the ability to internally debate and refine their approaches before presenting solutions. This is particularly valuable for complex problem-solving tasks that benefit from multi-step reasoning. We recently had the opportunity to present this work at the Cerebras Systems & OpenRouter Qwen 3 Hackathon, where it was selected as the 3rd winning project. More importantly, the plugin is now available as open source, enabling anyone to enhance their AI workflows with advanced reasoning capabilities. For those interested in the technical details, the implementation is available on GitHub at https://lnkd.in/g7nKqFt6, and I've created a demo video showing the plugin in action: https://lnkd.in/g2RwfqmC Excited to see how the community builds upon this work to advance reasoning capabilities in open AI systems. #ArtificialIntelligence #OpenSource #MachineLearning #AI #Innovation #TechLeadership

NotebookLM:"The cycle of scientific discovery is frequently bottlenecked by the slow, manual creation of software to support computational experiments. To address this, we present an #AI system that creates expert-level scientific software whose goal is to maximize a quality metric. The system uses a Large Language Model (LLM) and Tree Search (TS) to systematically improve the quality metric and intelligently navigate the large space of possible solutions. The effectiveness of tree search is demonstrated across a wide range of benchmarks. In bioinformatics, it discovered 40 novel methods for single-cell data analysis that outperformed the top human-developed methods on a public leaderboard. In epidemiology, it generated 14 models that outperformed the CDC ensemble and all other individual models for forecasting COVID-19 hospitalizations. By devising and implementing novel solutions to diverse tasks, the system represents a significant step towards accelerating scientific progress." https://lnkd.in/ei7RdSzm NotebookLM:"This academic article introduces the Interpolating Neural Network (INN), a novel architecture designed to bridge the gap between machine learning (ML) and traditional computational methods like interpolation theory and tensor decomposition. The primary motivation is addressing the challenges faced by conventional #ML solvers in computational science and engineering, such as poor accuracy with sparse data, limited scalability, and high computational costs. By integrating interpolation concepts and tensor decomposition (TD), the INN significantly reduces computational complexity and memory usage while maintaining high accuracy, effectively outperforming existing ML models and traditional partial differential equation (PDE) solvers." https://lnkd.in/e2e5nNfP NotebookLM:"This perspective piece advocates for a concentrated effort in NeuroAI, the intersection of neuroscience and artificial intelligence, as the key to achieving the next generation of #AI capabilities. The authors contend that while neuroscience has historically driven significant #AI breakthroughs, a renewed focus on fundamental research in NeuroAI is necessary to overcome current limitations in building intelligent systems. Central to their proposal is the embodied Turing test, which shifts the focus from human-centric skills like language to the sensorimotor capabilities shared by all animals, demanding that #AI models interact with the physical world with the competence of their living counterparts. Successfully passing this test, which is rooted in hundreds of millions of years of evolution, is presented as a crucial roadmap for the next generation of #AI, leading to more robust, flexible, and energy-efficient systems." https://lnkd.in/em2g8SsF

Optimization problems are common in various sectors yet they are often solved heuristically due to the specialized expertise required for more optimal solutions. Addressing this challenge, researchers from Stanford have introduced OptiMUS, a LLM-based tool designed to understand and solve linear programming problems directly from natural language descriptions. OptiMUS not only automates the development of mathematical models and solver code but also evaluates and refines its solutions, making advanced optimization techniques more accessible across industries. OptiMUS works by taking a natural language description of an optimization problem and transforming it into a structured format that it can understand and solve. Here's a step-by-step breakdown of how it does this: 𝟭. 𝗣𝗿𝗲𝗽𝗿𝗼𝗰𝗲𝘀𝘀𝗶𝗻𝗴: OptiMUS identifies key components from the problem's description, such as parameters, objectives, and constraints, and understands the context. 𝟮. 𝗕𝗿𝗲𝗮𝗸𝗶𝗻𝗴 𝗗𝗼𝘄𝗻 𝘁𝗵𝗲 𝗣𝗿𝗼𝗯𝗹𝗲𝗺: It uses a multi-agent framework to divide the problem into smaller parts, each handled by specialized agents for formulating math, writing code, and evaluating solutions. 𝟯. 𝗔𝗴𝗲𝗻𝘁𝘀 𝗪𝗼𝗿𝗸𝗶𝗻𝗴 𝗧𝗼𝗴𝗲𝘁𝗵𝗲𝗿: A "manager" agent coordinates the workflow, assigning tasks to formulation, programming, and evaluation agents based on progress. 𝟰. 𝗖𝗼𝗻𝗻𝗲𝗰𝘁𝗶𝗼𝗻 𝗚𝗿𝗮𝗽𝗵: OptiMUS employs a graph to track relationships between problem components, ensuring focus and efficiency by considering only relevant information. 𝟱. 𝗜𝘁𝗲𝗿𝗮𝘁𝗶𝘃𝗲 𝗥𝗲𝗳𝗶𝗻𝗲𝗺𝗲𝗻𝘁: The agents continuously refine their outputs, improving mathematical formulations, code, and solutions until the best outcome is achieved. OptiMUS revolutionizes optimization by automating the conversion of natural language into mathematical problems, making advanced techniques accessible to a wider audience. Its potential to improve decision-making, enhance solution quality, and expand the use of optimization across industries signifies a major step forward in both operational efficiency and AI-driven innovation. Paper: https://lnkd.in/eHzW9CPG

Reasoning Models 2.0, combine Reasoning with Tool Use! ✨ START teaches LLMs to use tools, such as code interpreter to improve reasoning and problem-solving. Self-taught Reasoner with Tools (START) integrates tool usage with chain-of-thought reasoning by enabling tool calls, self-check, exploration, and self-debug while reasoning using a self-learning framework. 👀 Implementation 1️⃣ Collect math problems (AIME, MATH) and coding tasks (Codeforces, LiveCodeBench) 2️⃣ Create context-specific hints like "Maybe using Python here is a good idea" 3️⃣ Generate tool-assisted reasoning data (insert hints after conjunctions like "Wait" and before stop tokens) 6️⃣ Score trajectories, remove repetitive patterns, and create a seed dataset with successful tool-assisted reasoning examples. 7️⃣ Fine-tune model on seed dataset, then self self-Distill to generate more diverse reasoning trajectories 6️⃣ Fine-tune the base model using rejection sampling (RFT) on the extended dataset Insights 💡 Improves math accuracy by +15% (AMC23: 95.0%) and coding by +38.6% on medium problems. 📈 Test-time scaling via sequential hints boosts AIME24 performance by 12%. 🐞 Code template modification reduces debug errors by 41% in training data. 💡 Adding tools (Python interpreter) improves performance more than adding more training data. 🧠 Large models already possess latent tool-using abilities that can be activated through hints. 🛠️ Two-phase training (Hint-RFT then RFT) allows the model to learn effective tool usage. 📍 Hint place selection is important. After conjunction Token and before stop token. Paper: https://lnkd.in/emF_m8Qz

I recently received a question about the tools used for the attached simulation. I previously highlighted that I´m using a full open-source workflow, but I didn't actually list the tools.   Some time ago, I regularly posted about open-source simulation tools, but I missed writing a summary for this CFD simulation. Here is the full list of tools used:   Salome Platform – Salome is a toolbox that includes geometry and mesh modules and can act as a GUI for some solvers. I have used Salome to generate a mesh from the input geometry and export a .MED file that can be read by code_saturne Code Saturne – is a CFD FVM solver that can handle several flow types and includes a variety of turbulence modules. Large simulations can be parallelized. BVTKNodes and Blender – Blender is a 3D modelling, animation and rendering tool. With the BVTKNodes plugin, it can also be used to visualize VTK solver outputs for stylized renderings. Paraview can be used for this purpose too, providing a more intuitive way to navigate the visual toolkit's filters and manipulators.   #simulation #visualization #engineering