SciPhyRL: a physics‑informed reinforcement learning method for cost‑aware portfolio optimization
This paper presents SciPhyRL, a new way to build trading strategies for large investors that learns from historical data while explicitly accounting for trading costs. The method casts portfolio choice as a continuous‑time control problem over an extended state that includes positions, asset prices, and cumulative trading cost. Instead of using standard trial‑and‑error training, the authors reduce the optimization problem to a partial differential equation (the Hamilton‑Jacobi‑Bellman equation) and solve a trajectory‑projected, pathwise version of it directly from data.
To solve that equation from offline records, the authors use physics‑informed neural networks (PINNs). By encoding the PDE into the training loss, the approach finds a value function and a stochastic policy in a single offline pass — avoiding repeated value or policy iteration. The paper also reforms the trading control from a continuous trading rate to a discrete target holding. That makes the strategy reach signal‑implied positions immediately while measuring execution costs with a microstructure‑based, quadratic price‑impact model.
The framework is explicitly distributional: it learns a risk‑sensitive policy from a fixed dataset generated by a behavior policy and gives a way to assess uncertainty in the learned policy. To make computations tractable, the authors approximate a partition function by a Gaussian mixture and derive a simpler semilinear PDE. They also develop a semi‑analytical solver with convergence analysis, and they report experiments on a 14‑asset ETF universe using an engineered oracle signal. In those tests, the learned Gibbs (soft stochastic) policy produced substantial out‑of‑sample improvements in Sharpe ratio versus static and myopic baselines, while keeping volatility and turnover under control.
Important caveats are highlighted in the paper. The method is offline and depends on the available historical dataset, so its performance hinges on how well that data represents future conditions. The numerical experiments used an engineered “oracle” signal, which is not the same as testing on real forecasting signals. The price‑impact model makes costs quadratic in actions and moves the problem outside standard linear‑quadratic control classes, which complicates analysis. The authors also note that a detailed numerical comparison between the PINN solver and their semi‑analytical solver is left for future work.