From the little that I've played, I've found getting out to spin really scrambles the mind. When it comes to pace bowling, there's a fear that the ball will hit you, and it will hurt. With spin, that's not really the case unless you love sweeping but most of the times the ball hitting isn't a major fear.
That's why when you keep getting out to spin, it really messes up the mind. You see the ball, most of the times you are even aware of the direction that it will take after pitching but still at times you just aren't there. That's when you feel like, maybe you just aren't good enough.
With fast bowlers things happen quicker and the muscle memory is the only thing that you can actually control. So even though fast bowling is scary, and if you get hit in the body or there's spice in the pitch and the ball is is hitting the bat but still hurting your hands, and you are wondering what you should do next just to survive, you aren't second guessing every ball.
I'd personally face a fast bowler on a green pitch rather than a good quality spinner on a rank turner.