When I first started trying out TDD and (more so when I started using the TestDriven.Net), I still spend a good chunk of my test development time stepping through the code line by line, checking the state of variable etc. It took me a while to kick the habit, but when you get stuck into TDD you won't be spending anywhere near as much time stepping through your code trying to work out what’s going wrong as you probably used to.
Why? Because the unit tests you write in an incremental fashion, build slowly on the last and in turn your confidence in the code is greater – you no longer feel the need to step through the code. When I run a test that fails now I can usually spot the issue relatively quickly without having to "step in". Those small TDD steps really pay off.
Don't get me wrong, I still step into my code when I get a curly one, but it's more the odd occasion now than the natural response.
Stay out of that debugger - it will save you a lot of time