63% of poll results are entirely made up

The Home Office published a report last week, entitled ‘Sexualisation of Young People.’ It was trailed on the radio, along with some of its radical recommendations, which include relegating ‘lad’s mags’ like Nuts to the top shelf.  It’s an entirely worthy subject, and as the mother of a young teenage boy and a preteen girl, I was pretty interested in what it had to say.

The report author, Linda Papadopoulis, states firmly in the introduction that:

This is not an opinion piece, the evidence and arguments presented within this document are not based on conjecture but on empirical data from peer reviewed journals, and evidence from professionals and clinicians.

Unfortunately, as I read through, I became increasingly distracted by the type of evidence being presented in the report.   Because I am a huge geek, I started going back to a few of the sources being mentioned, to see whether they were really saying what the report author claims they were saying.  

And right now?  I don’t really trust the report.  

Here are some of the things that make my spidey-sense tingle:

 1. Vagueness about method

There are references to focus groups, but no detailed information.  The main evidence gathered appears to be that provided by the organisations and pressure groups consulted, together with desk research. 

2. Imprecise use of opinion poll evidence

‘A recent YouGov survey found that 27 per cent of boys are accessing pornography every week,with 5 per cent viewing it every day.’

Which YouGov study? How old were the boys? How do you define pornography? (I’m not being deliberately critical – in a discussion of sexualisation, there is a huge difference in meaning if the boys are 16 compared to 10)

‘Almost half of children aged 8-17…’

Why is an 8 year old being lumped in with a 17 year old?  This makes me suspect that the sample is not big enough to support a more detailed breakdown.  I also want to know more about the methodology of a study that gathers this information from teenagers and children.

3. Uncritical use of ‘voodoo polls’

 
 

Surveys have found for instance that a high proportion of young women in the UK aspire to work as ‘glamour models’ or lap-dancers.

 

Do they indeed? That’s appalling.  The reference here is ‘Deeley, 2008.’  This tracks to an article in the Times, citing a web survey conducted by an internet TV company called Lab TV.  63% of 1,000 girls questioned by Lab TV apparently thought Jordan was a good role model.  I say ‘apparently’ because that’s as far back as I can track this one – unless Lab TV is the one belonging to the (American) National Defense Education Programme. Which I doubt.  Still, I bet it was a great sample!

Mind you, it’s really not the same question, is it?  Thinking someone is a good role model isn’t quite the same as wanting to grow up to be that person.

4. Drawing on US evidence in a UK report

In the US, the number of magazines targeting the teen market rose from five to 19 between 1990 and 2000.

And the point would be? In an argument about inappropriate sexualisation in UK culture, the state of the US market is not obviously relevant.

5. Argument from dramatic, unsubstantiated examples

Padded bras, thongs and high heeled shoes are marketed and sold to children as young as eight.

I do think that girls’ clothes are often inappropriate,  but as mother of an 8-year girl, I can’t say I’ve ever spotted these examples.  Princess sandals may count as high heels. I know the point that the writers are trying to make but I would rather see it done with more everyday examples.

6. Straight-out confusion

 

Before the mainstreaming of internet access, it was asserted that the average of first exposure to pornography was 11 for males.  However, research suggest that this age is now much lower.

 

There are two references quoted to support this statement.  The first one is a 1985 reference – the pre-internet one.  The second one is ‘Greenfield, 2004.’   I Googled this paper, which is actually evidence presented to a US government committee.  It in turn references various studies which tried to estimate the age when adults first remembered coming across explicit sexual material.  It doesn’t support the ‘much younger’ argument in any way that’s obvious to me.

As I said, this is a subject that’s close to my heart, as a parent who is constantly trying to negotiate these issues in an internet-enabled world.  My first reading of this report, though, is that it is indeed mostly opinion.  That’s a disappointment.  Opinion is very important in this area, but I think the shakiness of some of the references quoted really does the subject a disservice.

Learning to love the stacks

Books in a Stack, by austinevan on Flickr

Books in a Stack, by austinevan on Flickr

I’m just starting a new project that involves doing a literature search before getting stuck into interviewing.

I spent untold years doing ad hoc research, in a cab-style ‘take the next project’ sort of way, and I can’t tell you what a complete pleasure it is to be allowed to go and look at the literature.

It’s maybe peculiar to the places I used to work, but there was quite a disdain towards Doing A Bit of Reading, and a complete horror of exerting one’s database search skills with all the Boolean logic which that entails.

Not all projects need much beyond basic immersion in what went before, but I do projects which involve setting up a new Thing, like a framework, as well as gathering data. What I have learned:

4 benefits of full-on research:

1. Not reinventing the wheel. NOT REINVENTING THE WHEEL!!! So many, many things have been done before, some of them rather well.

2. Allowing time and space for you to really consider the issue Nuff said.

3. Constructing new knowledge Outside well-researched topics, typically you’ll end up synthesising information from a grab-bag of odd-looking books, papers, and commentary.  This is marvellous.

4. You can come up with theories and hypotheses! And then test them! Possibly just me…asking the right people the best questions is a luxury indeed.

And so on to …*drumroll*

7 tips for a successful search 😛

1. Join a university library. This can be hard work without current connections, but there may well be a great, relevant library that you can use.  Online catalogues look like the answer to your prayers but in practice can be expensive and difficult to access, depending on the subject area.  Use your academic or professional affiliations to get access to books and journals.  I use the Wellcome Library,  Birkbeck College,  and Cambridge University Library; I also have access to the British Psychological Society’s collection.  Wellcome has a great cafe.

2. Recognise that opinions are legion but facts can be sparse. Most literature reviews outside academia involve several Emperor’s New Clothes moments: typically, the absence of any evidence whatsoever for strongly-held popular opinions.

3. Web resources look infinite but actually aren’t. This is the never-ending Web Ring problem where you chase promising sets of hyperlinks that eventually loop around themselves in a giant spiral of mutual attribution.

4. Look in parallel fields for great answers. It is quite likely that your particular horrible problem has been discussed and perhaps even solved in a rather different field of operation. In my experience, teaching is a completely overlooked source of great information.  Take e-moderating for example: 90% of the problems online researchers face have already been researched, written up and peer-reviewed by academics working in teaching and online learning.

5. Find the official resources. Government sources are excellent and entirely free.  You can find entire datasets this way.

6. Identify the star papers and books as quickly as you can. In any field, there are some essential pieces of reading.  In a brand new field, one of the most useful things you can find is the high-level textbook.  These are your Little Helpers. Love them and treat them well.

7. Be flexible, don’t print out in too small a font, and carry paper, pens and a USB stick everywhere.