True, but it would at least test OP's algo in some bearish market conditions. If no backtesting was done then it has only been tested in the current bull market.
I completely disagree. Backtesting matters, especially in HFT. What would you rather have: an algorithm that has performed well on 5 years worth of historical data or one that has performed well in the past 2 weeks.
Neither; these are techniques applied on observed data in an obviously dynamic open domain. The data is uncontrolled - so we have no idea if it is representative of the current state of the domain theory - does it cover the distributions properly; we don't know. We know the domain is dynamic in that the world economy moves like around alot and we can't predict these movements, in the sense that most actors did not call the last substantial negative market movement. The domain is open in that new features can appear that drive the value of the variable of interest; for example the availability of property in China, new battery technology, a patent on nuclear fusion.
We do not have techniques that account properly for any of these things.
You may as well draw lines on charts and sell that advice, you'd probably get as good a hearing in any Investment Bank in the world !
He's not wrong, backtesting really is not the same as testing on real live data. You are fitting your models on data that has already been done, without any of the risks factored in. Even if you do find a system that picks every ups and downs in the backtest, it will do poorly with live trades because the volatility and risk is something that changes based on fundamental data.
However, in this case, the fact that no backtest or any sort of risk/reward ratios were published suggests high level of skepticism.
In my experience? The later. Regime change is a killer in trading algos.
Further, accurate back testing is very difficult for lots of reasons, the most obvious being your own actions aren't reflected.
Most algo firms use historical data to develop a hypothesis & then many different ways to validate that hypothesis. Test trading being the one with the most weight.
Does backtesting work better if you, say, start with seven years of data, use the first five to judge the strategy, and then test it against the last two? Is that a good way to avoid fitting the data?
A friend had a technique that worked very well on the NASDAQ 100 with roughly those BT criteria. One problem was overlooked: The NASDAQ 100 changes over that time. So you aren't necessarily investing in the same securities in year 6 as you were in year 5, even though you're investing in the same "top 100". IOW, backtesting in this particular case had future knowledge that wasn't easy to see.
When he started (paper) trading in the real NASDAQ 100, the returns were negative. In that he was lucky: They could have been positive for some time, until the constituent stocks changed.
reply