From what i can tell, this sequence is a special case of the Hardy-Littlewood Conjecture H (see https://arxiv.org/pdf/1004.0536). This conjecture states that every large integer $n$ that is not a perfect square is the sum of a prime and a square, and predicted an asymptotic formula for the number of such representations. If true, this immediately implies the list is finite.
The full conjecture remains open, but there is strong partial progress on bounding the set of exceptions (here and here). given how number theory is so connected, there's probably like 500249385 other niche conjectures that if solved would direclty imply this as well lol
The motivation for the conjecture seems to rest on a heuristic argument, where for some prime $q$ (taken to infinity), it gets increasingly many chances to be represented as $p + k^2$.
For an odd prime $q$, consider all $k$ from $1$ to $\sqrt q$. If $k$ is odd, then $q - k^2$ is even, so it can only be prime if it equals 2. If $k$ is even, then $q - k^2$ is odd and has a real chance of being prime. There are roughly $\frac{\sqrt{q}}{2}$ such even values of $k$. Each candidate $q-k^2$ is an odd number near $q$, so by the prime number theorem, it has roughly a $1\over\ln(q)$ chance of being prime (with some local correction factors). Hence, the probability that all $\frac{\sqrt q}{2}$ candidates are composite is approximately
$$\left(1 − {1\over \ln (q)}\right)^{\tfrac{\sqrt q}{2}} \approx \exp\left(\frac{-\sqrt q}{2 \ln(q)}\right)$$
This probability drops incredibly fast as $q$ increases, and a back of the envelope computation with mathematica shows that the expected total number of primes in of this form beyond 7549 is roughly 3.
however, despite being convincing, the issue (at least how i interpret it) with this argument is that it assumes being prime is a random independent event, but this isn't true irl ($q − 4$, $q − 16$, $q − 36$ etc are all correlated if you will)
Number theory isn't something im particularly good at so i could be mistkaen lol so take this as a grain of salt ig