by Rebecca Brueton
As we enter 2021, the power of science and technology to shape society continues to accelerate. And with it comes the potential to affect women’s lives.
Have you ever noticed that many modern feminist topics have a technological underpinning? Surrogacy and medical transition are such pressing issues in part because the technology behind them has become sufficiently advanced — and profitable — to be rolled out to the masses. Age-old feminist concerns such as porn and prostitution have been hugely affected by the advent of the internet, yet science and tech aren’t topics we talk about very often.
Writing for Trouble and Strife online back in 2014, Deborah Cameron said that she found the lack of science in the publication “both strange and somewhat worrying,” adding: “If feminists want science to be used in a way that serves women’s interests, and to hold it to account when it does not, we need to inform ourselves about what scientists are doing.”1
The vast processing power of computers has changed the way the world works. The tech industry’s ability to collect and analyse data and to make meaningful predictions is unprecedented. I wasn’t surprised by pay-day loans company Wonga’s astronomical fees, but I was astounded to discover that they used borrowers’ Facebook activity and friends list to decide whether to lend to them.
The data that modern society collects on individuals has huge potential. But how is it used? And how could it affect women?
To find out more, I spoke to Timandra Harkness.2 She’s a writer and broadcaster, a presenter of science shows such as BBC Radio 4’s Future Proofing, and a lapsed stand-up comedian. She is also fascinating to talk to.
The blurb for her book Big Data: Does Size Matter? reads: “Big Data knows where you’ve been and who your friends are. It knows what you like and what makes you angry. It can predict what you’ll buy, where you’ll be the victim of crime and when you’ll have a heart attack. Big Data knows you better than you know yourself, or so it claims. But how well do you know big data?”
Rebecca: So, what is big data?
Timandra: Basically, you get lots of different kinds of data about the same thing and put them together. The best example came from a brain scientist called Paul Matthews in London. I said, “You must be really excited about all the data from brain scans.” And he said, “Well, yeah, but that’s just one kind of data. What gets me excited is when I get the brain scans and the patient’s medical records and their postcodes. And then I get the weather records from where they lived. I put all those things together, and I ask a new question, one that wasn’t thought of when the data was collected.” In his case, it was about the relationship between how many hours of sunshine they got and the progress of [the patient’s] multiple sclerosis.
The processing of big data is often done by a computer program that we call Artificial Intelligence. You don’t necessarily explicitly give it every stage of what you want it to do. You can say, “We want it to sort brain scans into healthy and diseased, or into male or female, or into violinists and non-violinists. And instead of trying to work out the rules for it, we want the computer program to work out what the rules are.”
Of course, the downside is that you don’t always know what processes it’s going through to get those conclusions. Then you get this black-box phenomenon, where people say, “Ah, our machine learning algorithm has worked out that there is a 98% chance of you reoffending.” And you say, “Right. What was its reasoning to get there?” And they say, “Well, we don’t know. Because it’s machine learning, we actually have no idea how it came to that conclusion.” It’s a bit like being the parent of a teenager: you made it — but you no longer have any idea what goes on inside it or makes it do what it does.
It’s slightly over-mystified, because it’s not thinking in the way a human is. It’s still a mathematical process. All it’s doing is working out the optimum mathematical way to get as close as possible to the answer you want. It’s very good at pattern finding, spotting early signs of cancer in X-rays. It’s important not to overegg how brilliant it is; it’s very good at probability things, but not very good at causality, for example.
Why should we care about it?
Because big data is very much involved in so many things these days that affect us directly.
I think we’re starting to be a bit disenchanted with it and realize that it’s not magical; it doesn’t have mystical powers. You can’t just tip all the information in the world into the top of it and pop out answers that are objective, true and right. But nevertheless, it does affect us.
The way your data is used to make predictions and decisions about you is definitely changing. Some of the people who are now described as ‘data brokers’ used to be credit reference agencies — people like Acxiom and Experian, who now openly market the fact that they have a lot of data on you. Just like the brain scientist, data brokers collect data from all sorts of sources, put it together, and tag it all to you, so that they have this very multidimensional model of you as a person.
Does this happen on an individual level?
Yes, on an individual level. And then they can use that data for advertising services to target you, and maybe for financial institutions to make decisions about you. You are possibly not aware that some shopping you did two years ago or some survey you filled in or some interaction you had with an airline … all of this is being aggregated and put into a database about you held by a data broker.
Also, governments go to sources like Acxiom to get information on people. Political parties have gone to them to get data on individuals to use for targeting campaigns. So the idea that you can separate private, government and political is a complete myth. They’re all trading information with each other.
Does this concern you?
Yes, it does, because if somebody has a lot of data about me, and they’re using it to make predictions about me and decide what to offer me, and I don’t even know what information they have — then it’s a very unequal power relationship. This could really affect you. Are you going to get offered a mortgage? Are you going to be placed on a government watch list for potential terrorists simply because of some pattern of activity and who your cousins are and where you went on holiday and so on? These things do happen to people.
Or it could be as simple as you are looking at job adverts online, but you don’t see the advert for the really well-paid job at the tech company, because the program that places the adverts has been optimized for people who will click through that advert. If you don’t look to the computer like the kind of person who’s going to apply for a highly paid tech job, then you’ll never see the advert. Then, of course, this will perpetuate the fact — because you’re not going to apply for it if you don’t see the advert, are you? So, in ten years’ time, they’ll come back and say, “Well, people who look like you don’t tend to apply for highly paid tech jobs. So, we’re not going to show you the advert.”
And could “people who look like you” be women?
Well, quite. This actually has been the case in the past. Not deliberately, not because they wanted to exclude women, but because they were going on what correlates with the people who apply to these jobs.
Even if they program it to not ask about sex, they might leave in things like: how many years of in-work experience does this person have? What subjects did they study at school? What hobbies did they have? And all those things might correlate with what sex you are. You might have taken time out of work to have children, and that would cut down your number of years of continuous work. You might have been at a school which didn’t really think girls should study maths and it might be something you’d taken up later on. It might be that the computer has said, “Ah, well, loads of people who really like playing Dungeons and Dragons and extreme frisbee and skateboarding also go on to click these adverts. So, we’re going to especially show it to those people. People who liked netball and debating society and oboe club don’t tend to click these adverts, so we won’t show it to them.”
Wasn’t it a recruiting algorithm for Amazon that put women’s CVs to the bottom of the pile?
Yes, exactly. The algorithm was definitely not told “don’t recruit women.” But if the algorithm is just told, “Look for anything that correlates with the good employees we have now,” then it’ll find a lot of stuff which may not be remotely relevant to whether [the candidate] will be a good employee.
Is this something women should really be paying attention to?
Definitely. I think we should all feel that we don’t have to understand how data algorithms work to ask the basic questions.
When I wrote about the Ofqual algorithm [which was used to award UK exam results in summer 2020 after the COVID-19 lockdown, and faced fierce criticism for its statistical bias], I called algorithms “prejudice engines,” and a few people got a bit sniffy: “Oh, you’re discrediting all algorithms. It’s this particular one that was a problem.” But in a very real sense, any algorithm that predicts the future by using data from the past is a prejudice engine because it’s prejudging that the future is going to look like the past. That’s something we really have to guard against, because it takes away any idea that we can actually change what the future looks like.
One hundred years ago, when women said we should have the vote, a lot of people responded, “Don’t be ridiculous. Women can’t have the vote; they’re not educated enough; they don’t know enough about the world.” If you’d had an algorithm back then to say who should have the vote, the algorithm would have said: “Well, sorry we’ve looked at everyone and women are, on average, much worse educated and have very little knowledge of the world and no political experience. So, obviously, they shouldn’t get the vote.”
The very broad problem with handing over decision-making to algorithms is that it hands over human agency. It lets people off the hook of being responsible for their own decisions because they program their preferences into an algorithm and then say, “Oh look, the algorithm did it.” But it also takes away our agency to change the future, because it assumes the future is going to look like the past. And if you look back over the last 100 years, even 50 years, I think we’ve proven that the future doesn’t have to look like the past.
We tend to think of computers and algorithms as dispassionate and objective. But actually, these things are human errors of judgement, aren’t they? Algorithms are created by humans.
Absolutely. Everything about it depends on what humans have decided. The humans at Ofqual decided that their priority was to produce an overall national pattern of results that looked like previous years’ results. They said, “We want the overall pattern of results to look like the last three years. So, we will get your teacher to rank you in order. And then we will allocate the grades that we’ve already decided your class is getting in order of those ranks.”
That was absolutely a conscious and deliberate decision. Somebody asked them what guarantees they had that the algorithm wouldn’t discriminate against different ethnic groups or against girls or against state schools. So, they went away and tweaked it.
They went to all this effort to show that whichever sub-population you belong to, it would be fair to you as a sub-population. But it wouldn’t be remotely fair to you as an individual, which to me is really telling. If you were a kid who was bucking the trend at a below-average school, all the extra work that you put in counted for absolutely nothing. Which I think is extraordinary.
Is this our definition of fairness? That you can statistically show that no subgroup with a protected characteristic has been disadvantaged? This throws up a much bigger question with algorithms, which could be a positive thing, because you have to spell out: what are you trying to do here? What does fairness look like to you?
Should women be demanding that tech companies consider fairness? Do they need prodding?
Yes. I would like to see fairness much more discussed, because I don’t think it’s a simple question at all. I think it’s a really complicated question. What do you want? Do you want equality of outcome or do you want equality of opportunity? And what does that look like exactly? Bearing in mind that if we were already in a fair society, we wouldn’t even need to talk about this.
They’re debates that need to be prioritized, because I don’t think there’s going to be one easy answer. But if we start making that part of the discussion, then we take away this idea that fairness is just a mathematical idea, that you can program it into a computer and the rest of us can stop worrying about it.
How do you know if you’re the victim of an unfair algorithmic decision?
Exactly. You don’t.
This is one of the problems. It does very much happen behind the scenes. This is why we need to start being a bit more pushy about saying, “What’s the basis of that? You don’t have to give me the code. I don’t want to have to learn computer coding to know how you did that. But I would like to know the criteria on which you decided this.”
That’s going to start opening some cans of worms. Because if you ask, “On what basis did you charge me more for this plane ticket than that person?” then if they’re really honest, they’ll say, “Well, we charged you more because we thought we could, because you were on an iPhone and this other person was on a cheap throwaway Nokia.” At the moment, the algorithms literally do that. “What device are you browsing our website on? Oh, you’ve got an expensive Mac! We’re going to up the prices that we show you.” So, if we start saying, “Sorry, what basis was that on?” things could get tricky, because there are all sorts of unfairnesses programmed in behind the scenes.
Will we see more people in the street with signs saying ‘Fuck The Algorithm,’ like the students protesting the Ofqual results?
That really did make me laugh. I had some surprises in 2020; I did not expect that!
Actually, I hope so, but I hope that it’s for the right reasons. I hope it’s not people saying, “I didn’t get the result that I wanted. So fuck the algorithm!” I hope people start to say, “Hang on, that algorithm just assumed that I would not do any better than anyone at my school had ever done before. And so it hardwired in low expectations for me and people like me.”
I wonder what impact it has on the design process that women are so underrepresented in STEM. In Silicon Valley, for instance, it’s still the case, isn’t it, that most of the designers are blokes?
It’s more narrow than that. Most of the people designing things are young, highly educated blokes who have been to one of the good universities, probably in Silicon Valley. And now live in California. It’s an incredibly narrow range of life experience.
It’s tricky because there’s a danger that they say, “We must be more diverse,” and what they want is for the photos and the website to look different: different-coloured faces, some women, and maybe somebody with a visible disability. And it becomes a box-ticking thing where, if you look diverse, that’s meant to be enough. Whereas what you want is a diversity of ways of thinking and diversity of approach. And that is, I think, really hard to legislate for.
To get that, you want a culture that genuinely likes to challenge itself and be much more open to different points of view. And, in some ways, we’re actually moving away from that at the same time as there are all these pressures to have visible diversity. The idea that you might genuinely have a range of points of view, and a good, healthy argument about what you’re going to do … people are retreating from that because they think, “Oh no I don’t want to say anything controversial or offensive, then I’ll be cancelled or banned.” So I would like to see a culture of dissent, I think, more than of diversity. If you encourage dissent, then you encourage people to raise problems and different points of view. And obviously it helps if you also have different life experiences.
Do you remember the Microsoft chatbot called Tay from 2016?
Yes. They put it out into the wild, so that it interacted with people on social media. And people soon realized that it was basically learning from them, so they deliberately taught it to be racist, I think.
That’s it; Microsoft pulled the bot from Twitter within 16 hours as it had become so horrifically racist and sexist. What you were saying about encouraging a diversity of ideas reminds me of Microsoft’s apology: they promised that the company wanted “to work towards contributing to an Internet that represents the best, not the worst of humanity.” A laudable aim—but the devil, as always, is in the detail: who gets to say who “the best” are? Similarly, Twitter currently claims to be “committed to creating a culture of trust, safety, and respect,” yet misogynistic abuse on the platform is so bad that Joanna Cherry QC questioned Twitter in front of the Human Rights Commission on why they’re not protecting women’s human rights; and feminists are being banned from the platform in large numbers.3
It’s clear that Twitter is highly biased in what it deems acceptable or unacceptable, because a lot of women and also men get thrown off for saying — in my view — relatively uncontroversial things, like you can’t become a woman just because you say you’re a woman. You know, obviously, you’re at liberty to disagree with those things. And a healthy argument is a good thing. But I can’t say that they are particularly hate speech by any sensible measure. And then other people are left on to be really threatening, let alone harassing, or to post photographs that are arguably obscene.
Twitter is making some very interesting choices about who to ban. They’re certainly laying themselves open to the accusation that you can say anything you like as long as it’s to a woman. But you can’t say what you like if it’s to certain other categories of people.
I think this is a big general problem: social media can be a very unfriendly and unhelpful place for having public conversations. I think there are some people out there who genuinely can’t bear the idea that anybody has the temerity to disagree with them. Or [who have] inner prejudices about exactly who should definitely not have the temerity to disagree. But I don’t think the answer is to try and ban those people.
Can’t we just opt out of social media?
Oh, you can. The difficulty is that we need to relate to other people and a lot of the way we do that now is through technology. So you can opt out of it, but then you are to some extent opting out of social interaction.
I think it’s a real problem that we’ve kind of devolved our public space to tech companies. And that means that we’ve handed over the policing of public space to tech companies. I’m quite concerned about the current White Paper about Online Harms 4, because it seems geared up to put the pressure on tech companies to police public space — I mean virtual space, but public space, public discourse and public debate. And it’s very much put the onus on them to prevent harm, which is an incredibly widely couched term, [especially] if people are going to claim psychological damage from the use of certain words. You’re basically inviting unelected private companies to police public speech on a widely drawn basis.
We have no recourse to them to say, “You are having a chilling effect on free speech,” but they are at risk from authorities of being punished for allowing things to happen. We can all see that the trend from that would be for everybody’s freedom to interact to be controlled, but while so much of public life is now happening in these online spaces that could be a really bad thing for public life.
For example, many people are saying that we mustn’t allow anyone to be anonymous online because a lot of trolls are anonymous; they can give people a hard time and yet they are subject to no comeback because they’re anonymous. But, by the same token, I think a lot of people turn to anonymity precisely because it would be dangerous for them in some way to voice their opinions under their own name. I mean, just think of all the people who’ve been hounded out of their jobs for voicing unpopular opinions. You can absolutely see why somebody might want to use a pseudonym online.
Probably the impulse behind it is very good. But I don’t think the answer to it is to put all public space online under very tight statutory control under the guise of protecting us all from harm.
And I don’t think anyone should feel that they need a degree in computer science to have an opinion on this. Because if you say, “I’m on this chat room under a pseudonym, because actually I like to be able to sound off about my husband without him reading it,” you don’t need much tech expertise to look at that and go, “I don’t think it’s good to take away my right to anonymity here.”
We’re generally not very good at collecting data on women, are we? As Caroline Criado Perez’s book Invisible Women: Exposing Data Bias in a World Designed for Men (2019) shows, women have been consistently ignored in scientific studies, or they’re not asking the right questions and bias against women is built in. What if we could potentially use machine learning to analyse huge datasets taken on a population level from new sources? Could that go some way to helping us to get past this?
That’s a really interesting question, because it can go both ways. It can be that we get separated out and treated differently when that’s not relevant. But then when it would be relevant, like actual medical conditions where there’s a real biological difference, researchers fail to make a useful separation.
Yes, I think potentially data could help. This is the thing with data and algorithms. It’s very easy to constantly find yourself talking about how they can go wrong, how they can be used badly, how they can erode your privacy and so on. But actually, there is massive potential to use them for things that we really couldn’t have known before — putting together data that’s already there and using it in a different way.
What else do you think feminists should be thinking about? What might creep up on us if we’re not looking?
[Laughs] I don’t feel remotely qualified to say what feminists should be doing.
I guess the easiest thing for people to focus on without thinking, “Oh, lord, now I have to learn algorithms” is to keep an eye on people who are using algorithms as a way of not taking responsibility for decisions that they are essentially making themselves. With the Ofqual algorithm, they went, “We want this to happen and we’re going to build an algorithm.” And then when everybody’s unhappy with the results, we’re going to say, “Oh, it was the algorithm.” It’s like, “Yeah. But you wrote the algorithm.”
One of the things that worries me is that data and algorithms are used, not only as a way of taking away the agency of the people about whom decisions are made, but also as a way of ducking responsibility by people who are taking decisions. And it’s very difficult to roll back from that.
I always think about the judges in the US system who are using a risk algorithm. They have a prisoner in front of them and they have a risk score that the algorithm has produced. You are the judge, so the decision is down to you. If the machine tells you somebody has got an 80% risk of violently reoffending, how easy is it for you to say, “I’m going to believe you when you say that you will not reoffend and you want a fresh start in life”? If you do that, and then they do go and reoffend, somebody is going to say, “That judge let them out on the streets to attack my son. And the computer had told the judge that they had a high-risk score and she chose to ignore it.”
Once the algorithms start to make these decisions, or even recommendations, it’s very easy to go along with them and quite hard to go against them. And yet then, where are we left as humans in terms of taking responsibility for what we do?
So, I think we should never accept “that’s what the algorithm decided” or “that’s what the data told us” as enough of an answer. I think you can say: you were led by the data — and what did the data show you? But what were your assumptions when you chose which data to look at? What data did you not have? It’s not enough to say, here’s a great big wadge of data and that’s science! What questions did you ask at the start, when people came along with their programs and predictions?
Do you think the solution would be cultural or legislative? Because, I’m guessing, either there needs to be a culture of people asking themselves such questions or they need to be compelled to do so, one way or another.
It has to be cultural, because you can put rules in, but if we all relax and go, “Oh, it’s going to be fine because there’s regulation” then it’s a bit like transparency. I think transparency gets a bit fetishized: “It’s alright, it’s transparent! So, nothing bad could be happening.” Ha! That’s not how it works. Just being transparent is no guarantee of anything.
But regulation is good. It’s very important to have a regulatory framework for the police, for example. The police can download data off your mobile phone, which at the moment they’re doing with very little legal framework. You don’t even have to be accused of a crime; this happens even if you’re a victim or a witness.5
I’ve heard of women being asked to hand over their phones when reporting sexual assault.
Yes, if they report they’ve been raped or sexually assaulted, then the police say, “All right, we’d like to take all the data off your mobile phone.” Would you think it was okay if you said, “I’ve been raped” and the police said, “Okay, but we’ll have to come in and search your house just to check you’re not lying”? You might reply, “Shouldn’t you have a warrant for that? I’m not convinced this is proportional.” And if you think about what’s in your mobile phone, there’s probably way more private information on your phone than in your house.
So I do think we need regulatory frameworks, but we also all need to have a culture of asking questions and not think, “I don’t really understand how algorithms work, so I’m just going to keep quiet.”
Sounds like we need to not be intimidated by numbers. And have some healthy skepticism.
Exactly. “Okay, you have a very flashy model, but I’d just like to know, what assumptions went into it? What data do you have and what data do you not have?”
Before you go, please tell us a bit about your book.
My book is called Big Data: Does Size Matter? Obviously the implicit answer is ‘it’s what you do with it that counts.’ I said to people, “I’m doing a thing about big data.” Lots of my friends, who are all smart, curious people, sidled over with some furtive joy and said, “Oh, good. What is data? I just keep hearing this word and I feel it’s really important and now I’m too embarrassed to ask. But now you can tell me.”
Rebecca Brueton has been trying to escape marketing for years, but keeps getting tempted back into it by friends with interesting projects. Rebecca credits her mother with introducing her to feminism; once she became a mother herself, she began to appreciate how much work there still is to be done. One of the founding members of feminist campaign Let Toys Be Toys, Rebecca fell down the gender-critical rabbit hole via Mumsnet a few years back and hasn’t found the bottom yet …
1 See Deborah Cameron’s entertaining review of Sarah Richardson’s Sex Itself: The Search for Male and Female in the Human Genome.
2 The interview has been edited for clarity and length.
3 If you’ve not yet seen youtu.be/S3X0LXARUTE, YouTube, 2 May 2019, it’s well worth a watch.
4 For a clear guide to issues with the Online Harms White Paper, see Graham Smith, ‘Online Harms White Paper: A Tale of Two Committees,’ INFORRM’s blog (The International Forum for Responsible Media), May 2020.
5 Bill Goodwin, ‘Data protection watchdog calls for controls on police mobile phone stop-and-searches,’ Computer Weekly, 18 June 2020.
