Neural Networks Learn Physics Without Being Taught

Neural Networks Learn Physics Without Being Taught

There is a strange thing happening inside some of the most powerful artificial brains ever built. They are learning the laws of physics without ever being told what those laws are. Not from textbooks. Not from equations. Not from a single lecture on conservation of energy or the Navier-Stokes equations that govern fluid flow. They are figuring it out on their own, from data alone.

This sounds like science fiction. It is not. It is a quiet revolution happening inside a subfield of machine learning called Physics Informed Neural Networks, or PINNs. And the implications are stranger than most people realize.

The Problem With Teaching Neural Networks

For a long time, neural networks were trained the way you might train a student to recognize birds: show them thousands of pictures of sparrows and finches, correct their mistakes, repeat. This works. But it is inefficient. A child can learn to recognize a bird after seeing a few examples. A neural network needs millions. And even then, it does not actually understand what a bird is. It just got very good at pattern matching.

The deeper problem is that neural networks are fundamentally ignorant of the rules of the universe. Show a standard neural network a video of a ball being thrown, and it will learn to predict where the ball goes next. But it will not know why the ball follows a parabolic arc. It will not know that gravity exists. It will not know that momentum is conserved. It will simply memorize the statistical relationship between past positions and future positions, and hope that pattern holds.

This works fine until you show it something it has never seen before. A different ball. A different throw. A different angle. The neural network fails because it never learned the underlying physics. It only learned the surface.

What Physics Informed Neural Networks Actually Do

In 2022, a team of researchers led by Salvatore Cuomo, Vincenzo Schiano Di Cola, Fabio Giampaolo, and Gianluigi Rozza published a comprehensive review of a new approach that solves this problem in a fundamentally different way (Cuomo et al., 2022). Instead of training a neural network to predict outcomes, they train it to satisfy the equations that govern those outcomes.

The idea is deceptively simple. Standard neural networks learn by minimizing the difference between their predictions and the actual data. PINNs learn by minimizing two things at once: the difference between predictions and data, and the difference between their predictions and the laws of physics.

Here is how it works. Imagine you want to model how heat spreads through a metal rod. The physics of heat diffusion is described by a partial differential equation, or PDE. This equation is a mathematical statement of a physical law: the rate of change of temperature at any point is proportional to the curvature of the temperature profile around that point. It is not a guess. It is a known, verified relationship.

A standard neural network would try to learn this relationship from data alone. It would need thousands of temperature measurements across the rod at different times, and it would try to predict future temperatures from past ones. It would never know about the PDE. It would just find a pattern.

A PINN does something different. The neural network itself is designed to output a function that satisfies the PDE. The network is penalized not just when it makes a wrong prediction, but also when its predictions violate the physical equation. The PDE becomes part of the network's loss function, a term that the network must minimize during training (Cuomo et al., 2022).

This means the network is not just learning from data. It is learning from the structure of reality itself.

The Architecture of a Physics Aware Brain

The researchers describe a multi task learning framework. The neural network must simultaneously fit observed data and reduce something called a PDE residual. The residual is a measure of how much the network's predictions deviate from what the physical equations say should happen.

Think of it as a student who is graded on two things: getting the right answer on the test, and showing that their reasoning is consistent with the laws of physics. If they get the right answer by cheating, they still fail the second part.

The network architecture is not fundamentally different from standard neural networks. It uses the same building blocks: layers of interconnected nodes, activation functions that introduce nonlinearity, and gradient based optimization to adjust weights. But the training process is radically different.

During training, the network is evaluated not just at the points where data exists, but at random points throughout the entire domain of the problem. At each of these points, the network must produce a function that satisfies the PDE. This is called collocation. The network is essentially being asked to fill in the gaps between data points in a way that is physically consistent (Cuomo et al., 2022).

The result is a neural network that can make accurate predictions even in regions where it has never seen data. It extrapolates not by guessing, but by applying the laws of physics it has internalized.

Why This Matters More Than You Think

The Cuomo review identifies several variants of this approach. There is the vanilla PINN, which is the simplest form. There are physics constrained neural networks, which impose stricter adherence to physical laws. There are variational hp VPINNs, which use a different mathematical formulation of the PDE. There are conservative PINNs, which enforce conservation laws explicitly.

Each variant solves a different class of problems. But they all share the same fundamental insight: the laws of physics are not just descriptions of reality. They are constraints that any valid solution must satisfy. And neural networks can learn those constraints.

This matters because classical numerical methods like the Finite Element Method, or FEM, are extremely powerful but have limitations. They require the problem to be discretized onto a mesh, which can be computationally expensive for complex geometries. They struggle with problems where data is sparse or noisy. They are difficult to apply to inverse problems, where you are trying to infer the underlying parameters from observations rather than predict outcomes.

PINNs solve many of these problems. They do not require a mesh. They handle noisy data gracefully. They can solve both forward problems, where you know the physics and want to predict outcomes, and inverse problems, where you know the outcomes and want to discover the physics. The Cuomo review found that PINNs are more feasible than classical methods in certain contexts, particularly when data is limited or when the geometry of the problem is complex (Cuomo et al., 2022).

The Strange Emergence of Understanding

Here is where things get philosophically interesting. When a PINN learns to satisfy a PDE, it is not just memorizing a function. It is learning the structure of a physical law. The network's internal representations, the weights and biases that define its behavior, encode the relationships between variables that the PDE describes.

This is different from standard machine learning, where the network learns correlations. A PINN learns constraints. It learns that certain combinations of variables are not allowed. It learns that the rate of change of one quantity is tied to the spatial variation of another. It learns causality, in a limited but real sense.

The Cuomo review notes that most research has focused on customizing the PINN through different activation functions, gradient optimization techniques, neural network structures, and loss function structures (Cuomo et al., 2022). Researchers are trying to figure out what makes these networks learn physics most effectively. But the deeper question, the one that is not yet answered, is what the network actually knows when it has learned a PDE.

Does it know physics the way a physicist knows physics? No. It does not have conscious understanding. But it does have something that standard neural networks lack: a representation of the rules that govern the system. It can reason about what should happen, not just predict what has happened.

Where It Breaks Down

The Cuomo review is honest about the limitations. Despite the impressive results, theoretical issues remain unresolved (Cuomo et al., 2022). PINNs can be difficult to train. The multi task loss function can be hard to optimize, because the two objectives, fitting data and satisfying the PDE, can conflict. The network can get stuck in local minima that satisfy one objective but not the other.

There is also the question of what happens when the physics is wrong. If the PDE you encode into the network is an approximation, or if it fails to capture some important effect, the network will learn the wrong physics. It will produce predictions that are physically consistent with the incorrect equation, but wrong in reality.

This is not a flaw in the method. It is a feature. PINNs make the assumptions explicit. If you tell the network that heat diffuses according to a particular equation, it will learn that equation. If the equation is wrong, the network will be wrong. But at least you know what you assumed.

The deeper limitation is that PINNs require the PDE to be known in advance. They cannot discover new physics from data alone. They can learn to satisfy known physics, and they can infer unknown parameters within known equations, but they cannot invent new equations. They are not going to discover general relativity by watching apples fall.

The Open Question That Keeps Scientists Awake

The Cuomo review points to a tantalizing possibility. If neural networks can learn to satisfy PDEs, can they learn to satisfy any mathematical constraint? Can they learn conservation laws from data, without being told what those laws are? Can they discover the underlying equations of a system by observing its behavior?

This is the frontier. Some researchers are working on physics discovery networks, which try to infer the governing equations from data. These systems observe the dynamics of a system and attempt to find a set of equations that explain those dynamics. They are not given the equations. They have to find them.

The Cuomo review does not claim that this works reliably yet. But the fact that it is being attempted at all is remarkable. If it succeeds, it would mean that neural networks could help us discover new physics, not just apply known physics. They could analyze complex systems where the underlying laws are unknown, like turbulence, or climate dynamics, or biological networks, and suggest candidate equations for scientists to test.

This is not science fiction. It is happening in labs right now. But it is not proven. The Cuomo review makes clear that advancements are still possible, and that theoretical foundations remain incomplete (Cuomo et al., 2022).

What This Actually Means

• ▸You can train a neural network with less data if you teach it the physics first. The network does not need to see every possible scenario. It needs to see enough to constrain the solution, and the physics fills in the rest. This matters for problems where data is expensive or dangerous to collect, like nuclear reactor design or drug discovery.

• ▸PINNs can solve problems that classical methods cannot. When the geometry of a problem is complex, or when data is noisy, or when you need to solve an inverse problem, PINNs often outperform traditional numerical methods. The Cuomo review found them more feasible than FEM in certain contexts (Cuomo et al., 2022).

• ▸The network's internal representations are interpretable in a way that standard neural networks are not. Because the network is constrained by physics, its weights encode meaningful relationships. You can examine the network and see what physical laws it has learned. This is a form of interpretability that most machine learning lacks.

• ▸The method forces you to make your assumptions explicit. If you encode a PDE into the network, you are saying: I believe this is the correct description of reality. If you are wrong, the network will be wrong. But at least you know what you believed. This is better than a black box that gives you answers without telling you why.

• ▸The biggest open question is whether neural networks can discover new physics. The Cuomo review does not claim this is solved. But the path is clear. If networks can learn to satisfy PDEs, they can learn to find PDEs. That would change everything.