China has sparked fresh concerns over its surveillance of social media sites by unveiling an algorithm that will scan billions of messages to look for signs of people with “abnormal” or “suicidal” thoughts.
Using artificial intelligence and machine-learning capabilities, the system developed by the Chinese Academy of Sciences will automatically compare messages and comments posted on social networking platforms and online bulletin boards with key words and phrases thought to be typical of those used by a depressed person.
A message will be sent to anyone assessed as having suicidal or self-harm tendencies, and they will then be referred to local authorities in their province or municipality for therapy, including mental health and psychological counselling, the China Youth Daily newspaper reported.
Popular social messaging tools like WeChat and Weibo, which have hundreds of millions of daily users, will no doubt be prominent targets for the system. The operation has been trialled since July 2017 and will now go into full service.
Suicide was the leading cause of death for Chinese aged between 15 and 35 in 2017, according to statistics from the Chinese Ministry of Health. Authorities are claiming an accuracy rate of 92.2 percent from the 14,435 messages sent to users during the scheme’s trial period.
However, there are inevitable concerns that the algorithm will be misused in a country that often is accused on spying in its own citizens. Reports have not said what safeguards will be installed to prevent the system from being abused and to protect the privacy of users.
Critics have said there is an enormous risk of the scan parameters being widened to include the unmasking of people suspected of being dissidents or of spreading “politically inaccurate” thoughts online.
It is believed that censors and national security agencies already use key-word searches to monitor online traffic. Messages sent via apps like WeChat are reportedly kept for a period of time to allow checking, despite denials by the app’s developer, Shenzhen-based technology giant Tencent, that it built a backdoor access for the organs of state.