News & Commentary: 2005-02-08

Journalism vs Science, Statistics, Feedback and Honesty

Sensible Use of Statistics

Yet again, I find a link proving without a doubt that so many journalists have not the first clue about science and maths. They seem to be able to master the words but not the concepts. Just for the record, I'll outline where he has gone wrong.

It is, for example, pretty easy to demonstrate that innate differences between men and women impede the latter in many careers, not just science. Women have a statistically significant tendency to miss advancement opportunities because of childrearing.

You can't just say something is statistically significant unless you have some actual statistics to back yourself up. No such statistics are provided, nor is it clear how to go about collecting such measurements. For that matter, "to miss advancement opportunities" is quite a vague and ill defined criteria at best -- any real statistical study needs a clear criteria to work against. In short, journalists love to start with something that seems anecdotally plausible and create their own "statisticaly significant" results from thin air. Such results are nothing more than misleading lies.

most men are hopeless at math and science

Again a result with no foundation in measurement nor a clear criteria. What does "hopeless" mean in practical terms? The author is welcome to describe his own abilities but might be polite enough to refrain from denigrating other people, especially when talking from ignorance.

The fact that a group may be statistically better or worse at something tells one nothing about the abilities of any given individual.

This is also wrong, the statistics do give information about an individual -- they give us a probability distribution. Such probability is conditional on what else we happen to know about that individual. Ultimately, when we have perfect knowledge the probability distribution collapses and there is no more uncertainty... in the meantime (and in the real world) perfect knowledge is not available and we depend on probability.

To think otherwise is to misunderstand statistics and probability. A gambler knows that a card pulled randomly from a deck has a one-in-13 chance of being an ace - but any given card is either an ace or it isn't, which is why poker players actually look at the cards in their hands.

So that's all there is to poker -- look at the cards, no problem, anyone can be good at this game! Funny how not very many people are good at it... almost as if there is more to gambling than looking at the cards.

And so there is (duh!) because you can't look at ALL the cards, you can only look at the cards in your own hand and (hey presto!) that still leaves a lot of information missing (like the cards in the other guys hand, and the cards still to be dealt). Once more we come to good old conditional probability -- the cards in your hand provide you with some information that has an influence on the probability of what you might get dealt and also influences the chance of you winning the round... but chance still plays a part.

Similarly, you never have full knowledge of a person's abilities. When you first meet them, gender is something you can easily discover and since there is a statistical relation between gender and knowledge and skills, you can make some rough estimate as the the likelihood of their skills being useful to you for a particular application (or for that matter you can make an estimate as to how likely they might be to pass a university course). Asking this person to demonstrate some skills, or asking them to explain what they know will provide additional knowledge and thus narrow the probability to some particular conclusion. There is a formula for this process which is "Bayes Theorem" and the formula will never completely discard any input variable -- thus the gender still remains an influence over the final probability even when more information is known but the relative importance of the gender factor becomes smaller as other more significant information is available.

To prejudge a person on the basis of gender, race or ethnicity is like leaving your hand face-down on the table.

Quite the contrary actually, since perfect information is guaranteed not to be available, ignoring any factor that will influence the conditional probability is going to result in a misjudgement. Of course other factors of information beside gender, race or ethnicity should be considered when making a decision. Larry Summers never suggested that examination results should be ignored or that educational background was irrelevant. The implication that he ever suggested ignorance as a selection strategy is insulting to Summers and to the intelligence of anyone reading such an article.

In fact, it is the politically correct crowd that time and again refuse point blank to accept some measurements while drawing extensive conclusions from others. There are some measurements that we are simply not allowed to make, to protect us from the evil of being able to draw a conculsion from physical reality. Not only do they gamble with some of their cards face-down on the table but they jump up and down insisting that everyone else should also do so.

Arbitrary Redefinition of Mathematical Terms

In addition, and worse, a perceived lack of expectation might dissuade women from even trying to excel in science; prejudice, or even statistically significant correlations, can in this way become self-fulfilling. Pursuit of scientific truth is one thing, but one should be careful not to stoke negative feedback loops of this kind.

This one annoys me more than the others because feedback theory is very important in economics and in our understanding of the world. Any "self fulfilling" prediction is by its nature a positive feedback loop. That is to say, the loop gain is a nett positive. The mathematical analysis of feedback systems is well established and there is no room to be wishy washy on this issue. Negative feedback is a stabilising force that brings the system toward equilibrium whilst positive feedback is a destabilising force that drives the system away from equilibrium.

No doubt Peter Gordon feels it quite reasonable to define something as negative when it is something that he personally dislikes. Sadly, by attempting to remodel already established mathematical definitions he only displays his own ignorance and engourages ignorance in others. At the same time he is happy to relegate persuit of truth to second place (or possibly further back) and foolish ignorance to first place. Ignorance makes people easier to control and makes it more difficult for them to answer back, it also makes the world a more dangerous place, ultimately more prone to disaster.

The Practical World

However, there are good practical arguments against the practice: discrimination is economically inefficient. By refusing to rent to certain people, one reduces the supply of renters and thus rents. By refusing to hire or promote certain people, one will not have the best possible employees.

Let us consider various non-gender and non-racial forms of discrimination. If two people are asking to rent and one is dressed cleanly and tidily whilst the other is wearing tattered clothes and looks a bit dirty, which one is more likely to be offered the lease? I would guess that the tidy one will get it most of the time. Many people believe that someone who dresses nicely might also be someone who will look after property and keep a tidy house. There is no causal relationship between clothes and housekeeping but a great number of people believe there is a correlation (I doubt anyone has bothered to do a study on it but never the less, common wisdom is that the tidy dresser will also be better at looking after the house).

From a mathematical point of view, making a decision based on apparel is no different to making a decision based on other appearance-related issues such as gender or racial features. The fact is that a bad tenant can cost a landowner a lot of money very quickly and the change in price caused by reducing the number of potential renters is a small factor in comparison. Since a landowner always has limited knowledge of their potential tenants, they use whatever information they can go on, including appearance. Using this information does not guarantee a correct decision, but then again nothing else does either; at least they can slightly improve the probability of a correct decision.

From a political point of view, scruffy dressers don't make much of a pressure group and can't get laws passed to protect themselves from discrimination. As a consequence, some forms of discrimination (which are every bit as "unfair" as sexism or racism) are considered quite acceptable. Worse yet, there is a whole fashion industry making money by encouraging a belief in the correlation between clothing and every desirable trait imaginable. This industry does have considerable political power which guarantees that such discrimination will always be legal and socially acceptable.

Just like sexism, the "self fulfilling" nature of this belief will build a positive feedback structure that drives the system away from equilibrium and towards the extreme. Once this structure is entrenched, there is no easy way to push it back toward an equilibrium again.

In short, what Peter Gordon and friends must realise is that from a purely selfish point of view, the mathematics of the situation actually favour sexist, racist and other discriminatory behaviour on an individual basis. That said, from a social basis, the larger group of people does not benefit from such discrimination because it encourages conflict and division within the group. This is a situation akin to the "free rider" problem: when one person rides the train for free, that person gets an individual benefit but if everyone rides the train for free and no one pays for the train then the train stops and everyone loses.

The Honest Journalist

There is no such thing as a purely private commercial decision: agreements are backed and enforced by the law, and therefore society has the right to expect certain standards. When society condones anything other than objectivity and transparency in small relationships, it has a corrosive influence throughout.

This well-meaning suggestion has to take the cake after an extremely misleading article. The principles of "objectivity and transparency" would require Lawrence Summers to make available whatever information he has, regardless of the political approval rating attached to such information. The politically correct lobby is beholden to neither objectivity nor transparency and have demonstrated their interest in suppressing whatever information happens not to suit them.

I notice Donna Vestal's article in the Kansas City Star which is titled, Well, we can't say we find your honesty refreshing and this sums up the attitude of Donna and many others when it comes to information that they don't personally happen to like. I guess that at least we can say Ms Vestal is honest about her lack of regard for the truth, which cannot be said for Mr Gordon.

This article says it all really. The sad thing is that so few people believe it (especially the journalists themselves).

A Partial Apology

It is in the nature of apologies that they are hidden in obscure corners but I really did come across this rather better than average article only after reading a lot of junk on the same topic.

Virginia Valian points also out the self-fulfilling nature of discrimination and points out that most people make a statistical evaluation without realising that they are using prejudice as a source of information. She even cites her research references.

Unfortunately, this is such a difficult issue to research objectively because of the extensive political angles. Clearly the study she cites was designed from the ground up to get the conclusion that it did... why was the hypothetical job of assistant VP chosen to be in an aircraft company? Why not a plastic spoon company or chain of travel agencies? After all, your typical assistant VP has only vague knowledge of what really happens at the nuts and bolts level in any large company... they spend all their time checking stock options, scheduling schmooze sessions and checking their back for knife marks. What does the job of assistant VP have to do with research ability in maths and science anyhow? It is the postgrad students who do all the actual research and most of the teaching. The others are administrators, figureheads and political figures.

There is no doubt in my mind that the "aircraft company" was chosen exactly to evoke connotations of WWI flying aces, boys own adventures, Biggles and what have you. I tried to track down a bit more about this research, Madeline Heilman does have a web page here but her publications are listed as brief citations only (no article content). I don't know how the people interviewed were selected, where they were from, etc.

Anyhow, back to Virginia Valian's article:

The deep pockets of elite schools allow them to buy the services of a lot of very talented white men. They may be paying too much for those men, but they can afford it.

In this sense I quite agree, throw enough resources behind something and it doesn't matter what prejudicial factors are involved, you will be able to get results. The fundamental question here is, "why does the free-market fail to achieve an efficient outcome?"

Virginia Valian answers the question by pointing to Harvard's endowment. With a guaranteed large resource stream, their results don't have to be efficient, they just have to be something better than complete crap, which they always will be, short of utter mismanagement. But there is a deeper issue at work here... how do we measure any research output? No one wants to admit it but no one knows the answer. Once a group of men and women have passed whatever exams might exist and achieved some level of academic achievement, they go into research and teaching. How to objectively measure which is doing a better job? Research is too complex to have a clean measure of productivity. Some people's research only becomes valuable many years after their death, other people hit a lucky jackpot and stumble into something amazingly relevant. This can happen to anyone, male or female. Teaching is slightly easier to evaluate but at the higher levels it does get quite fuzzy trying to decide which crop of graduates are better qualified an why. Worse, there are political forces at work trying to avoid any standardised objective measure precisely because a lot of people with substantial power are not interested in such measurements existing.

The reality of the situation is that at the top levels of these institutions, it isn't what you know, it is who you know and more importantly who knows you. There really is nothing objective that distinguishes who becomes head of department and who is relegated to ordinary teaching and mid-level administration. These people are not connected to the "free market" in any direct way and the indirect connections are very indirect indeed.

The most important message, though, is that if we raise expectations of women in science and give them the resources they need, they will make it to the top.

Which is effectively restating the principle that if you put enough resources behind something, you will still get results despite any prejudice and discrimination involved. Of course, people who promote so called positive discrimination can't see that they are part of the same problem (causing divisions in society, feeding unrest and unfairness) as any other type of discrimination. While Virginia Valian seems easily able to recognise the self fulfilling nature of positive feedback, she conveniently forgets the positive feedback loop is not broken by positive discrimination, it merely pushes away from equilibrium in an alternative direction.

Creative Commons License
This work is licensed under a Creative Commons License.

Back to News Commentary Index