[This post is a rewrite and update of one I did a few years ago; conclusions are broadly the same, but the data is a lot better and my old blog is no longer online. I’ll be posting some new follow-up analysis soon, though.]

Relationships over time

A first step in studying relationships over time is to ask: what fraction of a people are in relationships? In particular, what would a graph of the percentage of people in a relationship at different ages look like? A logical guess might be something like this:

No one dates for ages less than about 10, say. (C’mon, grade 3s don’t date – they just pull each other’s hair.) Then the percentage slowly increases as relationships slowly stick before eventually plateauing out after age 60, say. (Let’s ignore death. For now.)

Thinking more, though, I wondered whether there would be more features to the graph – specific points where the percentage rises faster or slower or where, on average, people are actually breaking up more than they are getting together, so that we would see an actual dip.

I had two ideas: one such time might be after high school – perhaps relationships peak in Year 12 (age 17), then slowly fail as people grow apart, move, go to University, etc. Anecdotal evidence suggests that a lot of high school relationships fail after 5-7 years, which might mean a dip around age 23-25. Secondly, you could conceive a dip at around age 40-50 in the “mid life crisis” phase. A new graph might then look like this:

Of course, it might be that even if these effects were real, they could be too small to notice when averaged over enough people.

But how to test this? The Australian Census records marital status, but only as Single/Married/Divorced (or De Facto through Dwelling information questions). That’s not enough for want I want though. I could survey people, but that’s painful and hard to get a good sample.

Then, I stumbled upon a blog post (inspired by yet another post) that looked at the demographics of Facebook members. Turns out, this data is made available to potential advertisers – and so pretending to be interested in advertising my next party to the 9 million Australians on Facebook, I started gathering data about males and females in Australia and their relationship status. I’ve in fact been gathering this data for a couple of years…but that’s, literally, another post. (Others have looked at relationship statistics, but not by age, etc.)

Results follow after the break!

I looked up males and females in Australia in January 2011, who listed their relationship status as either “Single”, “In a Relationship”, “Engaged” or “Married”. (Anyone who selected nothing or “It’s complicated” are not included; see below for a discussion.) Numbers are good across all age groups, with somewhat more women than men:

Yes, I went with the stereotypical colours – don’t hold it against me! To compare ages, I looked at the percentage of men and women at each age listed as in a relationship. Let’s look at the women first:

The shaded red area is the percentage of women who are listed as not single for each age, and the lines show how relationships, engagements and marriages contribute to that. (We’ll come back to those dots!)

First of all, I was roughly right in my overall shape – it starts at low (though not zero – but we’re starting at age 13! And check out those 13 year old marriages. Uh huh, right), increases, then roughly plateaus.

Also really nice is that the peaks in the separate types of relationships are sequential: relationships peak at around 22, engagements at 27 and marriages from 35 onwards. Of course, you can’t naively assume that the individuals of that first relationship peak end up in the marriage column, but it does neatly reflect the different stages of life and love (nawwww).

My hoped-for post-high-school dip hasn’t appeared, nor has my mid-life crisis. If we were being overly confident about our data, you could squint and say marriages seem flat from 30-45, then slowly rise for later ages – possibly suggesting that divorces have slowly the net marriage rate in those years. This needs more investigation. Also, net relationships (red area) apparently fall even while marriages rise, possibly suggesting this is probably either margin of error or a reporting bias.

Let’s look at the men quickly before we move on to further discussion:

Again we have roughly the same peaks, and relationships, engagements and marriage are in the right order. The really flat marriage plateau seen for women doesn’t appear here, so it probably was just chance.

It’s also interesting to compare the two:

We see that women are about three years behind men in both total relationships and marriage, so men are probably marrying women around three years younger than them (either that, or there’s a few guys who are serious cradle snatcher sand are throwing off the stats!) The gap widens a little til early 30s, then shrinks – eventually age is no barrier!

Interestingly, for any age on my plot, a higher percent of women are married, or dating in general. While some of these are going to be same-sex partnerships or marriages, it should affect both about the same (unless I’ve unearthed an incredible statistic and lesbians are far more common than gays!) Census data shows that this is a real effect, however, and that it switches over sometime around age 60. You can’t see that on my figures, however, because Facebook lumps lumps all data for over 64s into the one category (oh, the discrimination!).

The census data shows that from age 65, more and more women start to become single – which I guess has a pretty morbid interpretation. Still, my data does hint at a boost in relationship numbers in later years – so our oldies are still finding love, if not marriage!

But is it real?!

Okay, let’s cut to the chase. We’ll only really know if this data is any good if we compare at least some of the results to an independent source. Say, the Australia Bureau of Statistics. After some rather painful googling, I found that marriage data by age is indeed available, in the form of the aptly titled data set “20680-Social Marital Status by Age by Sex – Time Series Statistics (1996, 2001, 2006 Census Years) – Australia”.

And, of course, anyone paying attention already knows the punchline – I’ve plotted the 2001 and 2006 census data points on the charts above. That’s a damn fine match! Women on Facebook slightly more likely to be married than the Census, men slightly less – this could be either a change in the last 4 years (it’s comparable to the 2001-2006 shift) or there could be a bias in the data. Let’s consider a couple of options to explain this deviation.

Some sources of bias

First of all, it’s not a truly random sample: it’s people on Facebook. On the other hand, “everyone’s” on FB these days (nearly half of Australia!), so it’s becoming increasingly good. But married women are likely to have kids, and so are more likely to be on Facebook to check up on them – thus producing a disproportionate number of married women. Should that apply to men too, though? And I could equally propose that Singles have more time on their hands and are more likely to join. Without any evidence one way or the other, I’m going to leave this one as a small source of error.

What about another type of bias: are people more likely to list one type of status over another? If you’re single, you might not want everyone to know. Or maybe you do, hoping to attract attention. Equally, most people in a relationship want to shout it to the world (disgusting, no?) – but after changing your status back to single for the first time, many people decide to list nothing. And then there’s those who simply have privacy concerns, and don’t make it available.

How can we tell? If a higher portion of single/non-single people are hiding it, you’d expect there to be some correlation between those not listing and the relevant status. So let’s see the graph for women (the male version is basically the same):

Between 60-70% of women list their status up to their mid-30s (which seems surprisingly high to me). This then decreases until only a third do by their mid-60s. You’d expect that if there was a correlation between relationship (or marital) status and privacy then we should see it here – but we don’t! When relationship status changes most, there’s no change in privacy. And when privacy starts to change, there’s no impact on status.

There could be some crazy, coincidental effect that (for example) marriage is actually growing and impacting privacy, hiding the marriage growth. But this seems a bridge too far for me, given the census data especially. Nope, the only thing influencing privacy (I say boldy, waiting to be disproved) is age. Neat!

Conclusions, part 1

So, conclusion: we can use Facebook (is there nothing it can’t do?) to investigate relationships through the ages (ha!). The graph looks roughly like you’d expect (or I’d expect; take your pick), with about 80-85% of people ultimately reporting themselves as being in relationships. By age 40, two thirds of people are locked into marriages and removed from the dating pool. And women are going for the older man, as pop culture would suggest.

I’ll shortly post a few follow ups. One looking at how this data has changed over the last 3 years (if at all!), and another looking at different countries (if you think India looks cool, you’d be right!) I also have a neat maths-y post where you can work out error bars for the status-listing bias, if not for the Facebook sample bias. After that, I can finally get into my newest dating data project!

Now I’m not saying the health care system is perfect, or denying that there may have been cases of fraud. But the alt-med movement has more frauds than you can poke a stick at.  And their biggest, that provably costs lives every year, is being investigated for the final time. And, turns out, it was all about the money.

Over 10 years ago, Andrew Wakefield fabricated a story that vaccines caused autism. From this burst the “antivax” movement, scaring parents into not immunising their children. Groups like the AVN are responsible for every death that results, so it’s not surprising they’ve recently been slapped down by the Health Care Complaints Commission (HCCC).

But that original study wasn’t just a mistake, it was outright fraud. And now we understand why – in part 2 of an investigative series, Brian Deer has revealed it was all about the money. The very thing that the alt-med movement complains about is at the heart of one of their most cherished campaigns. It’s a long read, but it’s worth it.

So will this cause the AVN and their ilk to change their tune? Nup. They’ll weasel their way around it – because unlike scientists, they won’t change their mind as new evidence comes to hand. But it’s still good to know the issue is put to rest once and for all.

The LA Times have performed analysis to produce a “value added” score for each teacher. This is basically using historic test scores for students to predict future performance and then comparing that to what the students actually did. E.g., if a student was in the top 10% and falls to the top 30%, it might be because of the teacher – or it could be just they had a bad year. If, however, this is true averaged over multiple students and multiple years, it suggests a problem with the teacher.

I’ve put a few excerpts below, with a little commentary, but I’ve posted my main thoughts on ranking teachers separately (have a read of that too!) The section “Study in Contrasts” is a particularly powerful pair of case studies, looking at two different teaching styles. One is seemingly effective, the other not, but there’s, arguably, little external incentive for teachers to improve or excel. As the reporters note,

Public school students are graded and tested all the time. Schools are scored too — California rates them in an annual index.

Not so with teachers.

Nationally, the vast majority who seek tenure get it after a few years on the job, practically ensuring a position for life. After that, pay and job protections depend mostly on seniority, not performance.

I absolutely agree with that – why should an old teacher be paid more than a young one if it happens that the young teacher is better read on modern education techniques, more enthusiastic, more knowledgeable and more proactive? (Hypothetically speaking, of course.) And, scarily, even if you identify the bad teachers and provide additional coaching, improvement is a slow process:

Bass acknowledged that it could take years for foundering instructors to improve, if they do at all. In the meantime, about 20 students a year will continue to sit through their classes.

“It’s tragic,” Bass said. “It means we’ve failed them.”

…

“It’s criminal,” Dixon said. “If you get a bad teacher in second and third grade, you’re doomed.”

Dixon has begun trying to remove the four teachers, a painfully slow process in California. It’s far more likely that they will feel the pressure and transfer to another school, she said.

“That’s not right,” Dixon said, “but it’s reality.”

That’s not to say that ranking teachers won’t be without difficulties. What do we do when parents all want the same teacher for their students? This is perhaps a hard problem, but that doesn’t mean we shouldn’t produce the data. Imagine if doctor’s statistics were readily available. Would you actually keep going to a doctor you knew misdiagnosed more patients than average? And again, the teachers’ responses are scary:

But many others say it would be impossible to accommodate every parent’s desire for the best teacher, and publicizing disparities would only turn one educator against the other.

Broadous Principal Stannis Steinbeck refused even to discuss the differences among her instructors, hinting at the tensions that might arise on staff.

“Our teachers think they’re all effective,” she said.

It’s that last quote that kills me. The Economist has a balanced analysis of the situation, and I’ll leave you with their closing quote:

There is no perfect way to evaluate teachers, but that is true of many jobs. (Should The Economist judge me on how much traffic this post gets? How much ad money it generates? How sharp the analysis is? Can that even be measured? How should each be weighted?) The problem is that the big teachers unions have not been credible participants in the conversation about reform, resisting efforts to incorporate test scores in the evaluation process, and fighting the consequences that must accompany bad evaluations. For its part, the Times plans to publish an online database with ratings for more than 6,000 elementary-school teachers based on test-score data. That is not fair to the teachers, who deserve a more comprehensive evaluation. But who is to blame for the absence of one?

Teachers who put in the effort and who are good at their job should be recognised both in pay and in title. I think (based, though, only on anecdotal evidence) that one of the big things holding back the teaching profession from getting better students going into teaching is that there is no opportunity to really “stand out” from the crowd, to get recognition for their excellence. There are, of course, a few awards, but ultimately your salary, in the short term, is a measure of your worth – in particular, your salary relative to your peers. We need performance based pay for teachers.

Not that this will be easy. Finding an appropriate metric will be frightfully complicated, and we would have to avoid a system that incentivises  ”teaching to the test” or even cheating [2a, 2b]. But that doesn’t mean it’s impossible – after all, if we can rank every student who finishes school (a score that determines their opportunities for the next few years!) sure we can do the same for teachers?

Two ultra-simple examples come to mind:

1. If Mr Smith and Mr Jones both teach the same grade at the same school, and have classes with random students, and every year for five years Mr Smith’s students do “better” than Mr Jones’ students – that’s got to say good things about Smith.

2. Every kid knows who the good and bad teachers are. Obviously, some kids hate teachers even though the teachers are good, but their opinions might still be worth listening to. And the students getting A’s and B’s are usually good judges – I could easily tell you which of my high school teachers deserve a raise.

I’m not saying this process will be easy, but that doesn’t mean we shouldn’t start. Part of the problem is that due to the opposition of the teachers unions (who are immensely powerful politically in Australia), any talk of performance based pay is quickly quashed. That means that we can’t have a discussion, we can’t gather any sort of data that might be used to investigate different measures – basically, we can’t make progress.

Part of the opposition comes, I’m sure, from teachers who know they’re not going to stack up. To them, I say boooo. But there’s probably legitimate opposition from teachers who worry about issues I’ve already mentioned – teaching to the test, metrics that value students’ short term gains over long term improvements, teachers who are inappropriately ranked (especially negatively). These are all good points – but again I say: the fact that something is hard doesn’t mean we shouldn’t do it.

In any system, some teachers who deserve to be rewarded won’t be, and some who deserve to be shot, won’t be. But in the current system, there’s no way to remedy that – if you haven’t been around for 40 years, that’s all there is to it, and there’s no point in trying to better yourself [3]. At least with a system featuring clear performance metrics (that are subject to ongoing review, research and debate) you’ll have an explanation, and the opportunity to do something about it.

Teachers need to be the most respected profession in our community. That means that teachers have to be prepared to accept recognition for both their successes and their failures and respond appropriately. The sooner we start, the sooner we can get right.

[1] “These people look deep within my soul and assign me a number based on the order which I joined.” Didn’t work for Homer either!

[2a] E.g., giving fire fighters a bonus for every fire they put out might not be the best strategy…

[2b] Have a read of Freakonomics for a great example of teachers cheating in their own students’ tests.

[3] At least as far as the system is concerned. Hopefully some teachers have the dignity to do better!

A British soldier left blind by a grenade in Iraq has told how his life has been transformed by ground-breaking technology that enables him to “see” with his tongue.

Lance Corporal Craig Lundberg, 24, can read words, make out shapes and walk without assistance thanks to a device developed in the United States which could revolutionize life for other blind people.

via Blind Soldier ‘Sees’ with His Tongue : Discovery News.

How amazing is our brain? It reminds me a little bit of touch typing, or learning to control a computer game by mouse/etc – after a while, you forget about the input device and just translate thoughts into actions.

My fantasy would be a USB port for your tongue (a clever acronym is left as an exercise to the reader…) that you could plug various sesnses into. Want sonar ability or to see in infrared? Want to know the current stock price? Want to know the health of your Starcraft army without looking? No problem! Just plug in the appropriate device and your brain will do the rest.