BLACK MIRROR PREDICTS THE FUTURE AS CHINA DECIDES TO RATE ITS CITIZENS

Imagine a world where many of our daily activities were constantly monitored and evaluated. Season three of Black Mirror starts in a dystopian world where people get rated according to the deeds they do on a daily basis. 

In June 2014, the state council of China published a gloomy-sounding document called "Planning Outline for the Construction of a Social Credit System". What if there was a national trust score that rated the kind of citizen you were? Just like the Black Mirror episode "Nosedive". Everything will be kept score of, what you buy at the shops and online; where you are at any given time; who your friends are and how you interact with them. It is not hard to believe or to picture because most of that already happens, thanks to the giant-data-collecting platforms like Google, Facebook and Instagram or even health-tracking apps such as Fitbit. 

Now put it into a whole new perspective, all these behaviours are rated as either negative or positive and broken down into a single number, according to rules set by the government. That would create the 'Citizen Score' and it would tell everyone around you whether or not you are trustworthy. Your ranking would be publicly rated against the entire population and would be used to determine your eligibility for a loan at the bank or a job, where your children can go to school – or even just your chances of getting a date.

The Chinese government is already in the process of developing the Social Credit System (SCS) to rate the trustworthiness of its 1.3 billion citizens. The policy states "It will forget a public opinion environment where keeping trust is glorious. It will strengthen sincerity in government affairs, commercial sincerity, social sincerity and the construction of judicial credibility."
Some citizens are less optimistic about the whole approach and its wider purpose. "It is very ambitious in both depth and scope, including scrutinising individual behaviour and what books people are reading. It's Amazon's consumer tracking with an Orwellian political twist," is how Johan Lagerkvist, a Chinese internet specialist at the Swedish Institute of International Affairs, described the Social Credit System.

Strictly speaking, participating in China's Citizen Scores is voluntary, but by 2020 it will be mandatory. The way in which each citizen acts, any legal person – whether you are a company or other entity – in China will be rated and ranked, whether they like it or not. 

Which companies will come up with the Social Credit System?

The Chinese government has given a license to eight private companies to come up with systems and algorithms for the social credit scores to two of the best-known projects. The first is with China Rapid Finance; a partner of the powerful social organization Tencent and developer of the messaging app WeChat with more than 850 million active users. The other, Sesame Credit, which is run by the Ant Financial Service Group (AFSG) and associated to the conglomerate, Alibaba. 

Ant financial sells insurance products and provide loans to small- to medium-sized businesses. The real star of Ant is AliPay, it is a payment arm that people use to buy things online, and to pay for dinner at restaurants, taxis, school fees, cinema tickets and even transfer money to each other.

Sesame Credit has also teamed up with other data-generating platforms, such as Didi Chuxing. The ride-hailing company that was Uber's main competitor in China back in 2016, as well as Baihe – the country's largest online matchmaking service. It is not hard to see how easy it is for a company like Sesame Credit to tap in to assess how people behave and then rate them accordingly.
 

So how do they rate people?

Individuals on Sesame Credit are measured by a score that ranges between 350 and 950 points. Alibaba will not reveal the "complex algorithm" it uses to calculate the numbers but they do reveal the five factors that are taken into account.
The first is credit history; do citizens pay their bills on time? Next is fulfilment capacity, which it defines in its guidelines as a "user's ability to fulfil his/her contract obligations".

The third factor is more personal, verifying personal information such as someone's mobile phone number and address, and the fourth category is where it get's interesting. Here the user's behaviour and preferences get measured, how consumers shopping habits play a role and what they decide to purchase online. Alibaba admits it judges people by the types of products they buy. "Someone who plays video games for ten hours a day, for example, would be considered an idle person," says Li Yingyun, Sesame's Technology Director.

"Someone who frequently buys diapers would be considered as probably a parent, who on balance is more likely to have a sense of responsibility." So the system not only investigates behaviour – it shapes it. It "nudges" citizens away from purchases and behaviours the government does not like.

The fifth category is interpersonal relationships. What does their choice of online friends and their interaction say about the person being assessed? Sesame Credit refers to it as "positive energy" online, kind messages about the government or how well the country's economy is doing will make your score go up.

Alibaba is adamant that, currently, anything that is negative and posted on social media does not affect scores (but this might be true, or not, we do not know, because the algorithm is secret).

But you can see how this might play out when the government's own citizen score system officially launches in 2020. Even though there is no suggestion yet that any of the eight private companies involved in the ongoing pilot scheme will be ultimately responsible for running the government's own system, it's hard to believe that the government will not want to extract the maximum amount of data for its SCS from the pilots.

Think twice before you post!

Posting dissenting political opinions or links to any badly influenced news has never been wise in China, but now it could directly hurt a citizen's rating. But there is the real kicker: a person's own score will also be affected by what their online friends say and do, beyond their own contact with them. If someone they are following or connected to online, posts a negative comment, their own score will also drag down.

So why have millions of people already signed up to this surveillance system? There might be a darker reason, unstated of sort – fear of reprisals – but there is also a lure in the form of rewards and "special privileges" for those citizens who prove themselves to be "trustworthy" on Sesame Credit.

The scores do indeed come with some advantages for the user itself. If their score reaches 600, they can take out a Just Spen load of up to 5,000 yuan (around £565) to use for shopping online, as long as it is on an Alibaba site. Reach more than 650 points, the citizen may rent a car without leaving a deposit. They are also entitled to check-in faster at hotels and use VIP check-ins at Beijing Capital International Airport. Reach more than 666 points you can get a loan of up to 50,000 (£5,700), obviously from Ant Financial Services.
Get above 700 and they can apply for Singapore travel without supporting documents such as an employee letter. And at 750, they get fast-tracked application to a coveted pan-European Schengen visa. "I think the best way to understand the system is as a sort of bastard love child of a loyalty scheme," says Rogier Creemers, a post-doctoral scholar specialising in Chinese law and governance at the Van Vollenhoven Institute at Leiden University.

Higher scores have already become a status symbol, with almost 100,000 people bragging about their scores on Weibo – the Chinese equivalent of Twitter – within just months of launch.
The citizen's score can affect their odds of getting a date or a marriage partner, because the higher their rating, the more prominent their dating profile is on Baihe.
Sesame Credit already offers tips to help individuals reach higher rankings, including warnings about the downside of friending someone who has a low score. This might lead to a whole different way of interaction between two individuals. People might have an incentive to say to their friends and family, "Don't post that. I don't want you to hurt your score but I also don't want you to hurt mine."

What about keeping the system secure?

Hackers could change or steal the digitally stored information, people might also be exposed to the birth of a reputation of black markets trying to sell under-the-counter ways to boost trustworthiness. The same way Facebook and Twitter sell followers and likes.

The government is attempting to make obedience feel like gaming, it is a method of social control dressed up in some points-reward system, basically gamified obedience.
Sesame Credit's chief manager warns people that the system is designed so that "untrustworthy people can't rent a car, can't borrow money or even can't find a job".
People with low ratings will have slower internet speeds; restricted access to restaurants, nightclubs or shopping malls, and the removal of the right to travel freely abroad with "restrictive control on consumption within holiday areas or travel businesses".

The scoring system will influence a person's rental applications, their ability to get insurance or even a loan, and even social-security benefits.

Individuals with low scores will not be hired by certain employers and will be forbidden from obtaining some jobs, including journalism and legal fields. Low-rating citizens will also be restricted when it comes to enrolling themselves or their children in high-paying private schools. This is the reality that Chinese citizens will face, as the government document states, the social credit system will "allow the trustworthy to roam everywhere under heaven while making it hard for the discredited to take a single step".

China's trust system might be voluntary, for now, but it is already having consequences. In February 2017, the country's Supreme People's Court announced that 6.15 million of its citizens had been banned from taking flights over the past four years for social misdeeds. The ban is being pointed to as a step towards blacklisting in the SCS. "We have signed a memorandum… [with over] 44 government departments in order to limit 'discredited' people on multiple levels," says Meng Xiang, head of the executive department of the Supreme Court. Another 1.65 million blacklisted people cannot take trains.

These systems can really descend into a nightmarish territory. The trust algorithms used are unfairly reductive, they do not take into account context. For instance, a person might miss paying his/her bills or a fine because they were in the hospital; another may simply be a freeloader. If life-determining algorithms are here to stay, we need to figure out how they can embrace the nuances, inconsistencies and contradictions inherent in human beings and how they can reflect life.
China's so-called 'trust plan' can be seen as George Orwell's 1984 meets Pavlov's dogs. Still, it is fairly straightforward, act like a good citizen, be rewarded and be made to think you are having fun. However, it is worth remembering, that personal scoring systems have been present in the west for decades.

Over 70 years ago, two men called Bill Fair and Earl Isaac invented the credit score. Today, companies use the FICO score to determine many financial decisions, including the interest rate on our mortgage or whether we should be given a loan or not. Majority of the Chinese people have never had a credit score, so they cannot get credit.
China's lack of a national credit system is why the government is adamant that Citizen Scores are long overdue and badly needed to fix what they refer to as a "trust deficit". The government also claims the system is a way to bring in those people left out of traditional credit systems, such as students and low-income households.

Is it possible that the Social Credit System is, in fact, a more desirably transparent approach to keep an even closer eye in a country that has a long history of watching its citizens?
"As a Chinese person, knowing that everything I do online is being tracked, would I rather be aware of the details of what is being monitored and use this information to teach myself how to abide by the rules?" says Rasul Majid, a Chinese blogger based in Shanghai who writes about behavioural design and gaming psychology. "Or would I rather live in ignorance and hope/wish/dream that personal privacy still exists and that our ruling bodies respect us enough not to take advantage?" Put simply, Majid thinks the system gives him a tiny bit more control over his data.

It is unnerving to even imagine that this system will be implemented within the next three years. Come to think of it we already rate restaurants, movies, books and even our doctors. Facebook is now capable of identifying you in pictures without seeing your face; it only needs your clothes, hair and body type to tag you in an image with 83 percent accuracy. In 2015, the OCED published a study revealing that in the US there are at least 24.9 connected devices per 100 inhabitants. All kinds of companies scrutinise the "big data" produced from these devices to understand our lives and desires, and to predict our actions in ways that we could not even predict ourselves.
In the US, the National Security Agency is not the only official digital eye following the movement of its citizens. In 2015, the US Transportation Security Administration proposed the idea of expanding the PreCheck background checks. The system wanted to include social-media records, location data and purchase history. The idea was scrapped after heavy criticism, but that does not technically mean it is dead. We are already living in a world of predictive algorithms that determine whether we are a threat, a risk, or a good citizen and even if we are trustworthy. We are getting closer to the Chinese system, even if we do not know we are.

Forget the right to delete or be forgotten. The future is where we will all be rated, data-mined and branded online. We are entering an era where an individual's actions will be judged by standards they cannot control and where that judgement will not be able to be erased. The consequences are not only troubling, it is permanent.

To trust the system we need to reduce the unknown; take steps to reduce the opacity of the algorithm. The issue with mandatory disclosures is that if you know what happens under the hood, the system could become rigged or hacked. But if humans are being reduced to a rating that could potentially impact their lives, there must be transparency in how the scoring works.

In China, certain citizens – such as government officials – will likely be deemed above the system. What will the public reaction be when their unfavourable action does not affect their scores?

It is still way too early to know how a culture of constant monitoring plus rating will turn out. Life will eventually become an endless popularity contest, with us all competing for the highest ranking that only a few of us can attain.