Obviously lower interest rates — ceteris paribus — do indicate the government should borrow more, but as Scott Sumner frequently reminds us: “Never reason from a price change.”
Most of all, it seems that rates, including long rates, have declined because of a flight to safety. The price change itself is ambiguous, but if you buy that interpretation we should be spending more, and borrowing more, to invest in safety.
Does infrastructure make us safer? Well, it depends. Public health infrastructure does, and that is where we should be spending more at the margin. I would expect that a lot of traditional physical infrastructure does not make us safer at all, and if you think its economic value covaries positively with gdp, it can make us less safe in the relevant portfolio sense.
Also keep in mind that a lot of marginal borrowing we do short-term and then refinance by borrowing short repeatedly, thus incurring ongoing interest rate risk. That is yet another reason to focus on actual safety rather than on outputs whose values rise sharply with gdp.
Many professional economists are getting this point wrong and thinking we should go longer on those bridges and tunnels.