By this time, I'd come up with the following theory of what was going on:
There are often several ways of thinking about the same thing. Some involve making and manipulating visual images. Some involve language.
When you're trying to think about something, you are actually using the bits of your brain whose real purpose is talking, or listening, or seeing (and reacting to the things you see).
If there's something going on nearby, it can activate one of these systems. For instance, if you see something flashing or moving that causes an alarm, that will refocus your visual system (all of it, including the brain part) to the suddenly interesting object. There's nothing you can do about this. An ancestor who was able to ignore things making sudden movements near him would one day ignore a snake. So he wouldn't be an ancestor of yours.
Similarly you can get alerts from your audio systems. Consider the famous cocktail-party effect, where you can be in a room with twenty conversations going on, and somebody in one of the conversations says your name, and suddenly you can hear that conversation above all the others.
What that means is that your audio system must be processing all the conversations, listening for interesting words, but not bringing them to conscious attention unless it detects something significant.
So even sounds that you don't realize are there are using bits of your brain.
So people thinking audio or language-type thoughts should do it much better in the quiet, and people thinking visual-type thoughts should do it better in the dark.
But worse than that, if you're thinking a visual thought and some sort of alert goes off (like seeing a movement, or something flashing, or a pattern being broken), then your entire visual system should be suddenly repurposed to assess the threat and YOU LOSE THE WHOLE THOUGHT.
And similarly for me, when I'm thinking about something, or reading, and some miserable bastard puts on the radiogramophone, that's it. Every so often something interesting (to my audio threat detection system) happens in the music, and my audio system focuses on it, and my stack of thoughts comes down and I have to build it up again. And once this has happened a few times, I am furiously angry and becoming averse to the very process of thinking.
Now this is kind of a weird theory, because it says that the programmers who don't notice ambient noise (and I would imagine this is all the programmers at github, because anyone like me would have managed to stand it for precisely one afternoon before walking out and never ever going back under any circumstances), are mainly thinking visual thoughts when doing their job.
Whereas the programmers like me (and I know a few examples), who don't notice the visual distractions must be thinking mostly language-type thoughts.
And this means that we are doing the same job in completely different ways. Which is kind of odd, if you think about it.
And there should be some people who can use either or both systems. I would imagine that those guys should be able to tolerate noise or visual distraction better than either Chris or I can, because they'll have alternative systems they can use, so they won't get completely stuck, but I would also imagine that they'd be much better at their jobs in a quiet and uncluttered environment where they can use both systems freely.
And if you are one of the people who uses only one system, then if you're able to learn to use the other one, you should become much better at what you do.
If you've got a theory, it should make predictions:
1/ An MRI scan of a man with music playing in the background should show audio activity
2/ There should be more activity if it's music with words.
3/ And even more if the words are in a language that the subject understands.
4/ Similarly for the visual stimuli, although you'd need to use an innocuous method of providing visual stimulation, and I can't think of any. Background music is ubiquitous and can usually be ignored. There isn't really such a thing as background television.
5/ Brain scans of people programming should show activity in visual areas if they identify as undisturbed by office noise.
6/ But should show activity in language-processing areas if they are disturbed by noise.
7/ Some people will show both or either. They should find that audio and visual noise reduce their productivity but do not disable them.
8/ People who are not disturbed by noise can become more effective thinkers by learning to think using the language parts of their brains as well.
9/ People who do not notice visual noise should learn to think visually.
10/ The best programmers will be driven up the wall by either.
Daniel Dennett says in 'Consciousness Explained', that he thinks that the inner dialogue starts when you use the bit of your brain which speaks to put thoughts into your own head via the bit of your brain that listens. Similarly he says that you can start off by drawing pictures to yourself and end up 'cutting out the middleman' and doing the same process entirely inside your head. Let's calls this sort of thing a feedback loop.
11/ There will be other modes of thought I haven't thought of, wherever there's a feedback loop to be exploited. Maybe some people think using touch? or emotion? Perhaps that's why some people seem to be 'naturally good at sport' or 'emotionally literate'.
12/ Maybe these feedback loops can be trained too?