63% of poll results are entirely made up

The Home Office published a report last week, entitled ‘Sexualisation of Young People.’ It was trailed on the radio, along with some of its radical recommendations, which include relegating ‘lad’s mags’ like Nuts to the top shelf.  It’s an entirely worthy subject, and as the mother of a young teenage boy and a preteen girl, I was pretty interested in what it had to say.

The report author, Linda Papadopoulis, states firmly in the introduction that:

This is not an opinion piece, the evidence and arguments presented within this document are not based on conjecture but on empirical data from peer reviewed journals, and evidence from professionals and clinicians.

Unfortunately, as I read through, I became increasingly distracted by the type of evidence being presented in the report.   Because I am a huge geek, I started going back to a few of the sources being mentioned, to see whether they were really saying what the report author claims they were saying.  

And right now?  I don’t really trust the report.  

Here are some of the things that make my spidey-sense tingle:

 1. Vagueness about method

There are references to focus groups, but no detailed information.  The main evidence gathered appears to be that provided by the organisations and pressure groups consulted, together with desk research. 

2. Imprecise use of opinion poll evidence

‘A recent YouGov survey found that 27 per cent of boys are accessing pornography every week,with 5 per cent viewing it every day.’

Which YouGov study? How old were the boys? How do you define pornography? (I’m not being deliberately critical – in a discussion of sexualisation, there is a huge difference in meaning if the boys are 16 compared to 10)

‘Almost half of children aged 8-17…’

Why is an 8 year old being lumped in with a 17 year old?  This makes me suspect that the sample is not big enough to support a more detailed breakdown.  I also want to know more about the methodology of a study that gathers this information from teenagers and children.

3. Uncritical use of ‘voodoo polls’

 
 

Surveys have found for instance that a high proportion of young women in the UK aspire to work as ‘glamour models’ or lap-dancers.

 

Do they indeed? That’s appalling.  The reference here is ‘Deeley, 2008.’  This tracks to an article in the Times, citing a web survey conducted by an internet TV company called Lab TV.  63% of 1,000 girls questioned by Lab TV apparently thought Jordan was a good role model.  I say ‘apparently’ because that’s as far back as I can track this one – unless Lab TV is the one belonging to the (American) National Defense Education Programme. Which I doubt.  Still, I bet it was a great sample!

Mind you, it’s really not the same question, is it?  Thinking someone is a good role model isn’t quite the same as wanting to grow up to be that person.

4. Drawing on US evidence in a UK report

In the US, the number of magazines targeting the teen market rose from five to 19 between 1990 and 2000.

And the point would be? In an argument about inappropriate sexualisation in UK culture, the state of the US market is not obviously relevant.

5. Argument from dramatic, unsubstantiated examples

Padded bras, thongs and high heeled shoes are marketed and sold to children as young as eight.

I do think that girls’ clothes are often inappropriate,  but as mother of an 8-year girl, I can’t say I’ve ever spotted these examples.  Princess sandals may count as high heels. I know the point that the writers are trying to make but I would rather see it done with more everyday examples.

6. Straight-out confusion

 

Before the mainstreaming of internet access, it was asserted that the average of first exposure to pornography was 11 for males.  However, research suggest that this age is now much lower.

 

There are two references quoted to support this statement.  The first one is a 1985 reference – the pre-internet one.  The second one is ‘Greenfield, 2004.’   I Googled this paper, which is actually evidence presented to a US government committee.  It in turn references various studies which tried to estimate the age when adults first remembered coming across explicit sexual material.  It doesn’t support the ‘much younger’ argument in any way that’s obvious to me.

As I said, this is a subject that’s close to my heart, as a parent who is constantly trying to negotiate these issues in an internet-enabled world.  My first reading of this report, though, is that it is indeed mostly opinion.  That’s a disappointment.  Opinion is very important in this area, but I think the shakiness of some of the references quoted really does the subject a disservice.

About these ads

16 Responses

  1. Really useful to have this kind of probing, clear, and detailed analysis of the report, a report which seems to be (understandably) attracting a lot of attention. Your critique works well because it is specific on what the report lacks, and that’s useful to me and other readers I suspect – thanks for sharing.

    • I found it really distracting – I kept twitching at the way references were being used which actually made it hard to engage with the argument.

      I’m sure there were plenty of sound references in there too, but generally it felt as though it was amplifying a given view very uncritically. And I say this as someone who really dislikes ultra-sexual culture (and lads’ mags); it just felt surprisingly journalistic for what was billed as a deep review.

      • Yes – surprisingly journalistic is quite right. I also really do wonder how this author came to write the report, since as far as I can tell she’s not particularly expert in the field.

  2. Hi Alison, thanks for a critical break-down of such an essential area of research – it is important that such reports are scrutanised so much because such claims can be very damaging. This is especially true when we speak of issues centred on children.

    Highly relevant analysis, considering the ongoing debates of how far to take police-checking and vetting of those who are in charge of vulnerable people – particularly children.

    Many thanks for the analysis.

    • See my reply to Phil – for me, the journalistic use of evidence really undermined the arguments that the report was making.

      There was also a lot of argument-from-worst-cases (for example, from studies of deeply-deprived youngsters) which I suspect also need to be treated quite carefully in a general discussion of culture.

      I absolutely didn’t expect to react like that either – I have a daughter of 8 who loves Lady Gaga and the Pussycat Dolls and goodness knows what that’s doing to her mental health…

      • Lady Gaga and the Pussycat Dolls

        Sounds like some sort of ironic supergroup.

  3. Good read, thanks. It echo’s my feelings, I tried to read the report as a factual document but came across far too many instances of evidence given without sourcing it.

    If you can’t show people where you’re getting your “facts” from then I’m afraid no-one should take what you’re saying as more than opinion.

    • I think I’d have been happy with opinion/argument and fewer (but harder) facts.

      It’s obviously an area where attitudes and opinions will matter as much as the data – but if you’re going to use data, for the love of everything, use good data.

  4. Thanks for this, I’ve been reading it too. I’m interested in the subject area, particularly gender and teen relationships, and I sympathise with *some* of its concerns, so I’m especially frustrated by its low quality. I agree with your points about the failings of the report.

    Thus far I’ve only got through the executive summary (trying to see if the points are properly backed up) but a sadly low proportion of the references seem to unproblematically say what the report implies they say. I don’t know if executive summaries are traditionally particularly heinous for misrepresentation – I mean, of course they’re making the fundamental points in the punchiest way, but that shouldn’t exclude them from basic standards.

    Probably the worst offender I’ve found so far was on video games: “The link between violent content and aggression has been cited in several studies and it is widely accepted that exposure to content that children are either emotionally or cognitively not mature enough for can have a negative impact.” The slipperiness of this sentence looked particularly suspicious (cited… widely accepted?), and lo, one of the three “several studies” was in fact a book surveying research on the effects of video games and concluding that there *wasn’t* a link between violent content in video games and aggression. Which is just startlingly disingenuous use of evidence.

    • Ah, fascinating (and what a brilliant user name). I wondered about the videogame evidence, but didn’t have time to go into it. I don’t actually see a lot of sexualisation in the zombie shootouts favoured in our house…I worry about all sorts of other implications but that’s not one of them.

      I don’t see why the executive summary should be different – in many ways, you wouldn’t expect a summary to cite references in that way. And yet it’s really, really journalistic.

  5. Incidentally (apologies for double-post), for anybody who’s interested in the subject area and frustrated by this report, a similar report (on “Sexualised goods aimed at children”) was prepared for the Scottish Parliament in 2008, and looks as if it was quite a bit more measured and well-researched.

    http://www.scottish.parliament.uk/s3/committees/equal/reports-10/eor10-02.htm

  6. Thanks for this Alison, I found it really helpful. I’m still working my way through the Sexualisation Review and have similar concerns to the ones you have raised.

    I found your points particularly useful for anyone trying to reflect on this report since it raises many worries most of us will have (particularly if we’re parents). However if it’s going to inform policy or practice it does need to be robust, and I’m not sure how strong a review it is.

    I appreciate not everyone is experienced in critically appraising reviews and consultation documents so I’ve blogged about how to do this in relation to the UK Sexualisation Review, and also linked to several other reviews on the same issue by other countries (including the Scottish one Metal-eating arachnid links to above, which is a good example of reviewing this complex issue).

    http://www.drpetra.co.uk/blog/sexualisation-of-young-people-report-released-how-useful-are-the-findings-here%E2%80%99s-your-chance-to-find-out/

  7. Thanks for your comments. I found it extremely confusing, in fact, but that’s probably because so many little alarm bells were going off in my head.

    It just strikes me that the topic is ripe for moral panic, so I would have liked a clear-headed review of the topic, ideally with input from the general public. I’m sure all those groups consulted are extremely sincere and have valuable things to say, but they seemed to be coming from, how can I put it, a more extreme experience?

    I think I’d have been happier with it as an outcome of a political think-tank, rather than a review, but it clearly sets itself up as serious review and yet fails to reach that level.

  8. An interesting dissection of the review. This garnered a lot of media attention when it was published – it is a shame journalists didn’t approach it with the rigour you applied!

    The worse thing is that by using such weak arguments, policymakers weaken further what might actually be a very worthwhile policy.

  9. Trying to track down that 63% want to be glamour models figure.

    http://news.sky.com/skynews/Home/Sky-News-Archive/Article/200806413365755

    is another report from the time, which says it was commissioned by http://www.thelab.tv – which is now a godaddy holding page

    archive.org gets a WordPress domain, which was a premium-rate text pollster – ie you texted to subscribe, and then got sent texts with questions. So that’s a scientific poll, isn’t it?

    It also has (helpfully) a copyright on:

    © 2005 Buongiorno UK Ltd.

    Which gets you to http://www.buongiorno.com, which is an SMS and mobile internet marketing company.

  10. IIRC they cite a poll that said 40% of parents had seen stuff on sale that disturbed them, that they thought were age inappropriate, on display in shop windows. As Brook Magnanti points out on her blog, this means of course that 60% had not. So the majority of parents are fine with the status quo, yet *wrings hands* OMG SOMETHING MUST BE DONE!!!11one. Yeah, right. To use a phrase from Bruce Schneier’s blog on security, this is security theatre. It’s doing something, anything, to make a show of having done something, without waiting to analyse the situation properly. This is why we have such hassle taking shoes etc off at airports rather than adopting the more effective Israeli model for airport security. It’s a pervasive culture of knee-jerk responses to tabloid-driven issues.

Comments are closed.

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: