Prepare to have your mind blown by some crazy perspective… graphical perspective, that is.
io9 has a full (but short) explanation of why we see it this way. If you don’t want to go over, here’s the even quicker version. When you’re looking at the horizon, all parallel lines along your line of sight converge to a point (one-point perspective). All the rays from the Sun hitting the Earth are roughly parallel, so when the sun rises, you see those rays fan out in all directions due to the perspective. If you turn around and face opposite the rising sun, the rays are still truly parallel, but again, the perspective forces them to converge to an opposite point.
I wonder what you would see if you looked straight up into the sky. Our brains process the sky as a sort of shallow salad bowl (the black dots) instead of a hemisphere (white dots), so would you see some trippy ellipsoidal equivalent of lines of longitude?