Distorted to YOU isn't necessarily so for the computer. Perhaps you can't see a visage because it's at an edge of a fisheye image, but it's easy enough to correct in software, as I said before, the system already does it ALL THE TIME for the 360 view, in ways which are way, WAY more complicated and mathematically complex than just straight defishing a static picture of someone.
The cameras are fisheye /because/ that's the most optimal way of capturing as much of the field of view as you can possibly do with any optical systems. The software to 'de-fisheye' a part of it is well known as well.
I'm a landscape photographer, and one of my favourite lens is a fisheye!