This is absolutely the case, and one of the least understood qualitative benefits. You can never explain it to someone living in a frame house, because that's their baseline. But having lived in each, there's a tangible difference in how they feel...I'd use the word 'vulnerable' for the traditional house, particularly when it's windy or storming outside.
To put science behind that, I think it comes from the disparity between interior and exterior conditions. When it's gusting outside and the house creaks, you're connected to the exterior. When it's gusting and the house is motionless and noiseless, there's a dissonance that makes you aware of the strength of the house.
Same with temperature. When it turns cold, I feel that inside as it's happening. But in the insulated concrete house, interior temps didn't change for up to 36 hours, so the difference between inside and outside really registered. You come to appreciate those differences really fast!