Towards the Automation of Political Censorship
We are at a turning point in the long history of censorship. This turning point is that of private and automated censorship. It marks a radical break with the guarantees associated with freedom of expression that the democratic struggles of the 19th century left us as a legacy.
The democratic-liberal heritage trampled underfoot
The 1881 law on freedom of the press—now interpreted in the light of the European Convention on Human Rights—is certainly full of loopholes, and we too often tend to exaggerate its liberalism. Nevertheless, it remains a fundamental compass for freedom of expression, providing, for example, [special procedural rules] for judging abuses of this freedom after the fact and establishing the exclusive jurisdiction of the courts in public proceedings.
In recent years, this legacy has been trampled on. Protections associated with freedom of expression are receding everywhere: on the streets, in the press, but also and above all on the Internet. While this phenomenon is not new, the current context of (anti)terrorist crisis and the authoritarian drift that pits the government against social movements are contributing greatly to it. To take just one example that is relevant here, in 2014, the legislature decided that “apology for terrorism” on the Internet had so little to do with freedom of expression that it could be subject to secret censorship by the Department of Homeland Security. This development led, at the end of 2016, to the censorship of statements calling for the destruction of police vehicles, in response to the indictment of individuals arrested in connection with the protests against the 2016 labor law.
In the same 2014 vote, Parliament also ruled that the glorification of terrorism was not covered by the 1881 law and its guarantees, and was more appropriately dealt with under the Criminal Code. By circumventing the procedural guarantees attached to the 1881 law, this development led to immediate court appearances and [totally disproportionate][3] convictions for dozens of provocateurs in the aftermath of the January 2015 attacks.
From experimentation to legislation
This headlong rush is reaching a tipping point. Since 2015, the French, British, German, and US governments have been putting pressure on multinationals in the digital economy to police their platforms. From ministerial visits to Silicon Valley to “state-platform” summits on terrorism, Google, Facebook, and their ilk have agreed to collaborate in mass censorship of terrorist propaganda outside the judicial system.
After numerous experiments in recent years, notably by [Europol] [4] and the [French Ministry of the Interior][5], these new public-private arrangements are about to be crowned by the rapid adoption of a [European regulation dedicated to the fight against terrorist propaganda][6], presented last September by the Juncker Commission.
France is directly behind this text. On April 12, Interior Minister Gérard Collomb and his German counterpart [wrote][7] to the European Commission urging it to act quickly to present and adopt this text. The Commission complied, presenting its draft on the sly on September 12—the day the European Parliament adopted the famous Article 13 of the [copyright directive][8], whose obligations regarding automatic censorship were then the subject of controversy.
In its initial version, this draft [regulation][6] on combating terrorism consists of a few articles:
- Article 4 requires all Internet service providers (web hosts, email providers, etc.), regardless of their size or operation (not just large platforms), to remove content reported by the authorities within one hour, under penalty of significant financial sanctions.
- Article 5 also allows these same authorities to impose such censorship not on the basis of applicable national law (for example, in France, the Cazeneuve decree of February 2015 on administrative censorship of the Internet), but on the basis of the platforms' terms of use. European police forces will therefore be able to do what the French Ministry of the Interior or Europol, report hateful or terrorist content and demand its censorship on the basis of private law.
- Finally, Article 6 requires hosting providers to adopt “proactive” measures to combat terrorist propaganda (Article 6). As in the copyright directive, this actually involves deploying computer filters that scan all content posted online by users of a service in order to block content that meets certain criteria set by these companies. This is a form of prior censorship, implemented by private actors using automatic tools that can be configured in partnership with governments.
Automatic censorship soon to be widespread
Two things emerge from the April 2018 letter from the French and German interior ministers, which have since been confirmed by information gathered in recent weeks from various ministries.
First, the certainty that the regulation is a foot in the door, a common way of imposing controversial measures by invoking supposedly unassailable justifications—in this case, the fight against terrorism. As the interior ministers explain in this letter:
“It will then be necessary to extend the rules laid down to content of a child pornographic nature and to content relating to hate speech, incitement to discrimination and racial hatred, and attacks on human dignity, etc.”
Ultimately, this new censorship regime is intended to be generalized. All that will be needed is to review the parameters of these filtering tools to add new categories of content.
The second lesson to be learned from this letter from the French and German governments is that these automatic censorship systems developed by the major platforms will have to be made available to all the actors targeted by this text:
“Large companies [must] provide logistical support to smaller platforms, as the latter's resources are too limited to respond effectively to our injunction to quickly remove terrorist content.”
It is true that these systems are expensive: Content ID, the tool implemented by YouTube to detect possible copyright infringements in videos posted by its users, alone will have cost nearly $100 million to develop…
Macron in support
In early November 2018, during the Internet Governance Forum held at UNESCO headquarters in Paris, Emmanuel Macron [returned][11] to this regulation, a text he said he “fully supports,” hoping for “its rapid adoption before the European elections.” That is, in about six months, an exceptionally short time frame for a security text that establishes extrajudicial censorship for all Internet players operating within the European Union.
In his speech, Macron confirmed the validity of the concerns raised by the regulation. Assuring that the “regulation of illegal or undesirable content” is “the next frontier,” he was quick to distinguish between “two blocks”:
- “The first concerns the fight against objectively and seriously illegal content—terrorism, child pornography.” For this content, artificial intelligence is “mature,” the French president assured. It is indeed the purpose of the European regulation to generalize its use.
- “The second block includes content whose illegal nature is subject to interpretation, which often differs from one country to another—hate speech, defamation, harassment.” For the latter, Emmanuel Macron believes that artificial intelligence “is not yet sophisticated enough to distinguish irony or parody, or to recognize an abject undertone in polite speech.”
“Not yet,” so soon? This prospect was recently confirmed by Mounir Mahjoubi, Secretary of State for Digital Affairs: when announcing the plan to combat hateful content on the Internet on February 14, he spoke unreservedly about the prospect of subjecting “hate speech” to automatic censorship.
As for the reference to “undesirable” content, does this mean that the head of state intends to encourage the extension of these censorship systems to content that is legal but deemed harmful by the authorities? This is a project which, among many other things, directly contradicts the image of a politician who claims to embody the European “humanist axis” standing up against the fascistization of the continent…
Artificial intelligence, the Holy Grail of states
During the three days of the Internet Governance Forum, “artificial intelligence” (or AI) was on everyone’s lips. It is, in fact, the new frontier that is supposed to transform all bureaucracies. It is AI that, according to the expression heard repeatedly during the event, will enable surveillance and censorship to “scale up.”
The progress made in recent years in machine learning and automatic text and image processing is indeed the Holy Grail for governments. At the end of 2017, an advisor to Theresa May [explained][13] that if governments needed American multinationals to do the dirty work, it was because, after all, “these companies have the best brains in the world.”
These firms, which have been working together since June 2017 as part of a consortium called the “Global Internet Forum to Counter Terrorism,” are touting impressive (but hardly verifiable) results in automating the detection of terrorist content: YouTube claims that 98% of such content is detected automatically, while Facebook explains that 99% of content related to ISIS or Al-Qaeda is removed before anyone reports it.
For the time being, these automatic censorship tools often only perform an initial filter. Most of the censored content is actually “moderated” by service providers based in India, the Philippines, or Morocco, who judge whether texts or images comply with the platforms’ terms of use. “While waiting for the robots,” these censorship workers, who labor in deplorable conditions, are given unrealistic targets of around 2,000 images per hour, which means they have to assess each image in less than two seconds. In 2017, Facebook [announced][17] that it would increase its moderation teams from 3,000 to 7,500 people. At the same time, Google [planned][18] to increase the number of YouTube moderators to more than 10,000.
These figures remind us that, despite its rapid progress, automated censorship is still in its infancy. But given the billions invested in AI research and pressure from governments, its widespread use is now within reach. It is well on its way to establishing a new regime of censorship.
The merger of the state and GAFAM: updating old logic
If we think of the state not as a clearly defined entity (as lawyers do) but rather as a set of practices and a rationality that Michel Foucault referred to as “governmentality,” then it is clear that what these developments reveal is the incorporation of these private actors into the state. the co-opting of their infrastructure and the dissemination of their expertise in the processing and analysis of masses of data that are now crucial in contemporary forms of government. What we are witnessing is therefore a fusion, rather than competition between states and GAFAM seeking to replace governments.
This logic of co-opting private actors by the state is a recurring theme in the history of censorship. From the 16th century onwards, when the development of printing played a key role in the spread of subversive political and religious doctrines, the modern state in its infancy was already resorting to public-private partnerships in the surveillance and censorship of communications.
In France, at the beginning of the 16th century, Cardinal Richelieu formed an alliance with Parisian booksellers who demanded a perpetual monopoly on book publishing in order to crush the competition. They were granted virtually unlimited privileges in exchange for monitoring publications and diligently enforcing censorship regulations. This policy, which was soon extended to the entire kingdom, enabled some thirty printers and booksellers to control, as best they could, the production and distribution of books. This did not, of course, completely stifle clandestine publishing, but it did make it possible to control the political effects of printing to some extent, to strengthen absolutism, and to fill the coffers of a few businessmen.
When they first appeared at the turn of the 1980s, computer networks—and the Internet in particular—were seen as a radically subversive technology that challenged the forms of communication control that had become institutionalized within the modern state and transposed to the various communication technologies that had emerged since the advent of printing. They would soon lead to the massification of cross-border information flows, the relative anonymity of digital communications, the appropriation of these technologies by protest groups and marginalized groups in the traditional public sphere, and, more generally, the democratization of public expression beyond the small circle of people who had previously had access to traditional media. All of this destabilized the means of controlling the public sphere and, even more so, the sovereignty of states. The internet was perceived as a dangerous space. It was therefore necessary to rearm the police force in the public sphere or, to use the term that Nicolas Sarkozy had [borrowed][20] from the Chinese Communist Party, to “civilize” it (a term that was [recently used] [21] by Mounir Mahjoubi).
The solution is therefore in the process of being found, and it has taken less than a quarter of a century. Today, instead of a dozen or so printers and booksellers across the country, a handful of American companies have been co-opted by government bureaucracies dedicated to surveillance and censorship. Despite soothing rhetoric about “digital sovereignty,” governments prefer to strengthen the dominant position of these multinationals by forcing all digital players to buy their filtering systems. Those who cannot afford to do so or who refuse to become censorship auxiliaries—particularly those who still keep an independent, non-commercial, alternative web alive alternative web—will be forced to shut down.
The instrumentalization of the fight against hate speech
We are told about the fight against terrorist propaganda and hate speech. But it is now increasingly clear that this fight will be based on automatic censorship mechanisms that will remain truly “black boxes,” [22]," which are by definition secret and managed by opaque public-private partnerships. The latter may agree to stage a certain degree of transparency, a little self-regulation, and adjustments presented as guarantees—like Facebook, which [promises][23] today to set up a “supreme court” to receive complaints from users who are victims of its “moderation” policy. But they will never offer the guarantees provided by traditional courts. And for good reason: these mechanisms are designed to circumvent them. Automatic censorship of communications has the advantage for those in power that it is almost invisible, and therefore virtually unchallengeable.
We are told that we are fighting hate speech through Internet censorship. But in the more than 40 years since laws were introduced to punish expressions inciting hatred and discrimination, have we really made any progress in this area? In fact, intolerance is openly displayed on the internet, where it targets structurally discriminated groups. However, such abuse is also commonplace in traditional media, in political circles, and often in the practices of institutions.
We hear talk of combating hate speech. But it is becoming increasingly clear that what is also being targeted is the invisibility of any speech remotely associated with forms of political violence that could be perceived as legitimate. This is illustrated, for example, by case already mentioned, when at the end of 2016, in the name of the fight against terrorism, the Ministry of the Interior [ordered][2] two participatory websites to censor a statement claiming responsibility for the burning of a gendarmerie hangar, an act presented by the authors as a response to police repression (this decision was finally [overturned] [24] in early 2019 by the administrative court, which was exceptionally seized of the matter). More broadly, all dissenting political discourse could soon be subject to this secret censorship. The temptation for those in power is real, as evidenced by the recent attempts by the Ministry of the Interior to [remove][25] a photomontage that replaced the faces of General Pinochet and his henchmen with those of Emmanuel Macron, Édouard Philippe, and Christophe Castaner.If adopted, the anti-terrorism regulation would be yet another step toward the implementation of these new measures designed to render dissenting political expression invisible. Even if we limit ourselves to the category of glorification of terrorism, the censorship tools developed by digital multinationals in conjunction with the authorities could be applied to many militant activities: the meaning of the term “terrorism” is indeed [so vague][26], or when a definition does exist, it is so broad that it can be used to justify censorship of any expression in favor of offensive or violent political actions—including the destruction of property, sabotage, or the blocking of infrastructure.## Warding off the re-feudalization of the digital public space {.wp-block-heading}If we care about freedom of expression, if we care about preserving a space where power can be criticized, where we can debate and organize, where a little common sense can be built, these new forms of censorship must be kept in check.What is needed is to understand that a large part of the problem with hate speech on the internet stems from what has become the political economy of this network. The crux of the specific problem posed by the internet is the way in which large digital companies condition attention regimes: clickbait content, simplistic and unnuanced content that plays on negative emotions; “filter bubbles” that lock us in while giving the illusion of neutrality, and offer a false sense of security by also allowing individuals with radically opposing political identities to come and attack us on platforms that are open to the four winds.Faced with fake news and the hatred that also plagues the internet, those who claim to govern us are rubbing their hands with glee: the decay of the digital public sphere now allows them to legitimize the return of old control strategies. This is the whole point of Macron’s speech at the Internet Governance Forum. However, this is not inevitable. In recent weeks, La Quadrature du Net has begun to outline positions aimed at using the law to combat these centralized infrastructures that distort public debate, while promoting the emergence of federated community communication hubs where spaces for expression and conversation, and therefore part of the public debate, can be redeployed. The key idea behind these proposals is to break the attention regime set up by platforms, based on the collection of personal data and the prioritization of content for commercial purposes, within gigantic infrastructures designed to make users captive.The goal is to promote a new topology of digital public space that can protect decentralized discussion spaces that can be linked to each other, leaving everyone free to choose their communities and the limits placed on freedom of expression, thereby giving them greater control over their exposure to the conflicts that inevitably arise in the public sphere. The hope is that this will reduce the risk of unwanted interpersonal conflicts. The aim is to ensure that the social demand for censorship is less strong and that prior and automatic censorship no longer appears to be the only valid option; that the principle of judicial protection of freedom of expression is preserved.
In any case, the urgent task is to break the alliance between the police apparatus and the major digital infrastructure providers. It is to prevent the consolidation of the technical and legal foundations of a [control society][28] that imperceptibly imprisons us in an algorithmic iron cage.
_This text, [published] [29] on the La Quadrature du Net website, is adapted from a presentation at the symposium “Regulating freedom of expression in the name of public debate,” held at the Institut de Recherche Philosophiques de Lyon (IRPhiL) on November 29 and 30, 2018.
[1]: https://fr.wikipedia.org/wiki/Loi_du_29_juillet_1881_sur_la_libert%C3%A9_de_la_presse#Garanties_proc%C3%A9durales[2]: https://www.laquadrature.net/2017/09/25/censure_indymedia/[3]: https://www.lemonde.fr/societe/article/2015/01/13/apologie-d-actes-terroristes-des-condamnations-pour-l-exemple_4555102_3224.html[4]: https://www.numerama.com/magazine/33628-europol-amorce-la-censure-de-contenus-legaux-contre-le-terrorisme.html[5]: https://www.interieur.gouv.fr/Archives/Archives-des-communiques-de-presse/2015 -Press-releases/Meeting-with-major-Internet-operators[6]: https://eur-lex.europa.eu/legal-content/FR/TXT/?uri=COM:2018:0640:FIN[7] : https://edri.org/leak-france-germany-demand-more-censorship-from-internet-companies/[8]: https://www.laquadrature.net/2018/09/12/droit_auteur_12_sept/[9]: https://www.laquadrature.net/2018/11/14/censure-antiterroriste-macron-se-soumet-aux-geants-du-web-pour-instaurer-une-surveillance-generalisee/[10]: https://www.inventiva.co.in/2018/11/youtube-weve-invested-100-million-in-content-id-and-paid-over-3-billion-to-rightsholders/[11]: https://www.elysee.fr/emmanuel-macron/2018/11/13/discours-du-president-de-la-republique-emmanuel-macron-lors-du-forum-sur-la-gouvernance-de-linternet-a-lunesco[12]: https://medium.com/@mounir/action-plan-against-hate-speech-online-10d396bfe71e[13]: https://www.telegraph.co.uk/news/2017/09/19/google-facebook-twitter-told-take-terror-content-within-two/[14]: https://archive.fo/fap0Y[15]: http://www.slate.fr/story/165767/travailleurs-horreur-quotidien-philipines-moderateurs-contenu-choquant-images-videos-reseaux-sociaux[16]: http://maisouvaleweb.fr/attendant-robots-coulisses-intelligences-artificielles/[17]: https://www.theatlantic.com/technology/archive/2018/02/what-facebook-told-insiders-about-how-it-moderates-posts/552632/[18]: https://thehill.com/policy/technology/363242-10000-google-staff-set-to-weed-out-extremist-content-on-youtube[19]: https://journals.openedition.org/leportique/625[20]: https://www.lemonde.fr/technologies/article/2011/01/21/l-internet-civilise-histoire-d-un-concept-a-geometrie-variable_1468412_651865.html[21]: https://www.huffingtonpost.fr/2018/09/21/pour-civiliser-internet-mahjoubi-prend-lexemple-dun-bus-dans-lequel-on-parlerait-comme-sur-twitter_a_23534704/[22]: https://fr.wikipedia.org/wiki/Bo%C3%AEte_noire_(syst%C3%A8me)[23]: https://www.lemonde.fr/pixels/article/2018/11/16/facebook-lance-une-cour-d-appel-pour-contenus-controverses_5384155_4408996.html[24]: https://www.legalis.net/actualite/annulation-de-decisions-de-blocages-administratifs-de-contenus-en-ligne/[25]: https://www.nextinpact.com/news/107547-quand-office-lutte-contre-cybercriminalite-exige-retrait-dun-photomontage-visant-macron.htm[26]: https://www.laquadrature.net/2018/12/07/une-loi-europeenne-pour-censurer-les-mouvements-sociaux-sur-internet/[27]: https://www.laquadrature.net/2018/10/09/regulation-des-contenus-quelles-obligations-pour-les-geants-du-web/[28]: https://fr.wikipedia.org/wiki/Soci%C3%A9t%C3%A9_de_contr%C3%B4le[29]: https://www.laquadrature.net/2019/02/22/vers-lautomatisation-de-la-censure-politique/[30]: https://droitphil.hypotheses.org/activites/en-2018/reglementer-la-liberte-dexpression-au-nom-du-debat-public