Understanding and measuring the influence of certain characteristics on the quality of text-based digital deliberation can help us make better design decisions
Citizens want to make their voices heard by policy makers on important decisions that affect their daily lives. After running a series of online public policy evaluation experiments last year, we found that while our participation process allowed us to include the views of (tens of) thousands of citizens on complex topics such as coronavirus exit strategies or energy transition, we had not provided an online environment fully suited to the task and needed to rethink its design.
While this type of participation encouraged citizens to reflect on the information and they found it useful to know the trade-offs for the proposed solutions, few felt comfortable making a final decision themselves. Some people were more thoughtful than others and some found the issues too complex. It was also clear to us that communication skills do not come naturally to many people, especially online, and that this essential element of the policy evaluation process should be encouraged as part of the platform design.
We realised that for particularly complex policy issues, an online platform promoting deliberative ideals would be more appropriate. Last but not least, where participatory processes can be conducted on a large scale, deliberation requires a different scale to reach its full potential.
Deliberation requires participants to reflect, to engage respectfully with different points of view and to give rational reasons for arguments. It is particularly suited to complex or sensitive issues such as climate change, where there is great uncertainty and many different points of view. The ideal communication space for deliberation is one of openness, inclusion, trust, rationality and political neutrality. However, most online platforms do not live up to these ideals.
Here we discuss the different design features of theory and practice that impact on the quality of text-based online deliberations in particular, recognising that video interaction and the way in which text and video are combined come with other considerations.
Time: Synchronous or asynchronous discussion?
The choice between a synchronous or asynchronous environment creates a trade-off between a more 'real' discussion experience and a more reflective, inclusive, egalitarian or accessible discussion. Real-time chat or video is more spontaneous and dynamic, which helps to create bonds between participants. Asynchronous communication, on the other hand, allows more time for self-reflection, removes location or time restrictions and increases access for people with slower internet speeds. It is a way of ' levelling the playing field' for the more and less informed public. Some research indicates that asynchronous discussions are likely to produce better quality deliberations overall.
Privacy: Identification or anonymity?
The choice between identification or anonymity in digital deliberations creates a number of trade-offs. With anonymity, a more egalitarian environment is possible as people feel freer to express their honest views, even if they are unpopular. Harmful social dynamics are reduced and people remain more focused on the task at hand. Anonymity can also allow the participation of civil servants or people with obligations of neutrality.
There is a trade-off between user accessibility and an understandable and well-structured discussion. Most online discussions take place on easy-to-use conversation platforms, such as forums, although their ability to promote fair and transparent discussion is questionable. Messages organised temporally, rather than thematically, are more difficult to navigate and connect with each other and content tends to be repetitive. New platforms such as Kialo now allow discussions to be visualised and arguments to be mapped, which helps participants to clarify their thoughts and better connect information. These platforms may require training or supervision of users, but they counter sponsored content and promote a fair and rational assessment of alternatives. However, for complex issues with a wide range of perspectives, rigid pro/con structures may not be appropriate. Other options include mind maps or systems maps.
Moderation: man or machine?
Having an independent moderator can greatly improve the quality of any discussion, as they can enforce social norms. However, large-scale online deliberations are more difficult and resource intensive. Moderators also suffer from human bias, as well as time and location constraints. Automated moderation techniques are therefore an important new area of research. Machine learning techniques, NLP or algorithms can help moderators with tedious tasks and give a more equal voice to less willing participants. While these algorithms are certainly useful, we need to consider the replacement of human bias with the bias inherent in the algorithms of automated moderators. Transparency is essential.
In short, designers do not always realise the extent to which their own worldviews, opinions or assumptions are embedded in the tools they create. Understanding and measuring the influence of certain characteristics on the quality of text-based online deliberations can help us make better design decisions.
Co-authored by : Anatol Itten
This article is a translation by Open Source Politics of the article published on the Medium "Participo", an OECD publication. To read the original article by Ruth Shortall, click here.
Ruth Shortall, PhD, is a postdoctoral researcher at the Faculty of Technology, Policy and Management of the Technical University of Delft. Previously, she was a computer programmer. Her research focuses on methods of deliberative policy evaluation and she is particularly interested in design aspects of online deliberative environments.
Anatol Itten is a post-doctoral researcher in co-creation and mass participation of citizens in government decisions at Delft University of Technology. Anatol is also co-founder of the Disrupted Societies Institute, a think tank aiming to unravel the dynamics of social division and polarisation. He has advised the UN Climate Conference COP23 and the German Ministry of the Environment on stakeholder engagement and citizen participation.
Just as the architecture of a meeting room affects who can be heard, the design of our digital tools both offers and prohibits certain political possibilities
This article is a translation by Open Source Politics of the article published on the Medium "Participo", an OECD publication. To read the original article by Jessica Feldman, click here.
Recent deliberative democracy projects have shown us that humans are remarkably good at collaboration, empathy and collective decision-making, even with complete strangers. In these times of physical distance, can we use networked digital tools to continue and even expand these projects? Can they take us even further into a future where deliberative democracy 'goes global'?
One of the keys to implementing true democracy will be a careful connection between engineering decisions and political values. We need to think carefully about 1) how and when to use different tools, and 2) how to build them. In this post, I focus on this second question: How can we proactively design for the needs of deliberative democracy? Below I sketch some areas where engineering decisions will need to be made and mention some possible concerns and solutions.
An algorithm is an automated process. When we think about algorithmic governance and deliberative processes, two sets of questions arise. First, where and how do we use digital technology in the deliberative process? For selecting participants? For occasional votes within a meeting? To collect, or even rank, the proposals to be deliberated on? There are many possibilities and many pilot projects. Secondly, how should these algorithms be written? The code itself will affect the conditions of decision making, just as any political protocol constrains our options.
While face-to-face voting is uncommon, it may be necessary in the context of online voting. If the deliberation leads to a vote, should the public be able to see the voting tables in real time? Should the identity of a participant be visible during comments or voting? Digital tools make it possible to record, compile and present this data quickly.
At the level of the code itself, we need to decide whether it should be visible, and to whom. We can learn from the recent Iowa Democratic primary scandal, where a closed, privately designed application was used to report vote tabulations and a "coding problem" resulted in only partial data being reported. For code to be reliable, it must be public: transparent and open source, and funded by the people.
Privacy and security
Computer scientists are taught to evaluate the security of a system on the basis of criteria they call the "C.I.A.". - Confidentiality, integrity and accessibility. In other words, communications/data should only be seen by those for whom they are intended. Data must not be compromised or falsified, and communications and information must remain accessible to those who should be able to access it - without being blocked, denied or deleted.
This is perhaps the most pressing issue: as many decision-making bodies move online, using pre-existing tools, we need to take seriously the threat of conversation monitoring, metadata collection, 'zoom bombing', server crashes (e.g. a cyber-attack) and online vote hacking.
Finally, participants working from home may not be able to speak or vote as they wish. This is not to say that digital tools should not be used, but that they should be designed to be secure and robust. In the short term, democratic bodies need to be carefully advised on which tools to use and make strategic and perhaps conservative decisions on how to use them.
Digitisation beyond quantification
While many debates about digital democracy focus on vote counting, deliberative democracy is much more concerned with conversations and consensus. We need to think carefully about how digital tools can help facilitate this process, rather than replace it. Some tools, such as Loomio or the consul software, have been developed from consensus-based communities, with the idea of helping discussions throughout the process.
Deliberative assemblies have always provided the affective conditions for developing empathy, derived from time-tested traditions of listening. As we move online, we need to ask whether - and if - these experiences can be achieved using digital tools. If so, what tools are needed, and how are our practices changing? If not, what role should digital play in supporting 'in person'?
In answering these questions, we need to keep in mind three key concepts:
Path dependency :
Once an infrastructure or a tool is built, we get used to using it, we start to organise our activities around it and build new technologies on top of it. We have to design things with that in mind.
Open Source :
As an engineer once told me, "open source is honest source". The code that underpins our decision-making and deliberative procedures should be publicly available.
Participatory design :
The best way to build these tools is through 'participatory design', in which the communities that will use and be affected by the engineering are involved in every step of the decision-making and testing process.
One of the great achievements of deliberative democracy is that it has been evolving and testing non-digital codes and processes for (at least) thousands of years. It offers many protocols that can be drawn upon for imaging digital processes.
This article is a translation by Open Source Politics of the article published on the OECD publication "Participo". To read the original article by Marcin Gerwin, click here.
It may seem to conceive the unthinkable. An online citizens' assembly? One of the essential elements of a citizens' assembly is to create a space for people to meet face-to-face. This is where the magic of citizens' assemblies lies. So why go online?
Well, sometimes unimaginable situations arise, and you start to wonder: what if? Would it be possible to achieve high-quality collective learning, deliberation and recommendations using digital tools? My answer to these questions is - yes. Certainly it would be a different experience than face-to-face meetings. But it could work.
As far as I know, no online citizens' assemblies have yet been held. Although various forms of online deliberation have been experimented with, I am thinking here of transferring the entire process online with a randomly selected group representative of the general public, following the same phases of learning, deliberation and formation of collective recommendations as in a face-to-face citizens' assembly.
Digital skills training phase
I would start with a training phase of at least two weeks to make sure everyone knows how to use the equipment and software, how to join a meeting, how to turn the sound on and off - all the basics. It could also be a social time where people get to know each other, talk about everyday things and get used to having an online conversation.
To help people who do not have experience in using the Internet, personal technical assistants could be recruited (these could be volunteers). In some cases, it may be necessary to purchase appropriate equipment, such as tablets with LTE internet (as in mobile phones), so that people do not need to have a router at home. Since the cost of location or catering is eliminated, it may be possible to purchase good quality electronic equipment without increasing overall costs.
We could draw on more than 40 years of experience in online education to design the learning phase. For example, it could consist of online expert and stakeholder presentations and reading materials. It does not have to be live. People could watch or read when it suits them best. Presentations should be relatively short, about 12 minutes.
To encourage learning, facilitators could offer offline or fun tasks, such as making a list of the most interesting things people have learned or filling out entertaining charts related to the material. Then, a facilitated study group call could allow participants to share their learning. As a general rule, all calls should be relatively short - one hour, maximum 1.5 hours, if participants agree. They could be held 3 to 4 times a week, and could last about two months (depending on the issue and its complexity).
Live group calls could include question-and-answer sessions with experts and stakeholders, during which participants could break into small groups to discuss the material before reconvening in plenary (this "chat room" feature exists on Zoom, Jitsi and other similar platforms). These small groups could follow the same good practices as those held in person, with 7-8 people per group, plus a lead facilitator and co-facilitator.
For the deliberation phase, the key is small group conversation. One possible option is for facilitators to gather ideas from the small groups and share them with the other groups, ensuring that knowledge is spread evenly. Draft recommendations could be developed in the same way. All recommendations could go through the same process of analysis, considering issues such as: what are the pros and cons; what are the costs; who would be responsible for their implementation; and other related trade-offs.
Since people have different reading preferences and some prefer a physical copy of long documents, the draft recommendations and accompanying analysis could be printed in booklet form by the organizers and given to participants for personal reflection before decisions are made.
The final step is to find common ground to finalize the collective recommendations. In the citizens' assemblies I have organized in Poland, this is usually done through a mixture of discussion and voting(see details here). This phase can be carried out online, by filling in electronic ballots or by using one of the existing collective decision-making tools.
Will I have confidence in the results of this process? Yes, if it was well designed and facilitated. Would it be the same as a face-to-face citizens' assembly? No. Nevertheless, it is worth a try because the current crisis situation, and any ensuing crisis, are precisely the kind of times when citizens' voices need to be heard loud and clear, in a meaningful and democratic way.
Marcin Gerwin, PhD, is a Polish specialist in deliberative democracy and sustainability. He designs and coordinates citizens' assemblies. He co-directs the Centre for Climate Assemblies and is the author of "Citizens' Assemblies: Guide to Democracy that Works".
This article is a translation by Open Source Politics of the article published on the Medium "Participo", an OECD publication. To read the original article by Mauricio Mejia and Claudia Chwalisz, click here.
As part of our work on innovative citizen participation, we have launched a series of articles to open a discussion and gather evidence on the use of digital tools and practices in representative deliberative processes. This work builds on the forthcoming OECD report Innovative Citizen Participation and New Democratic Institutions: Catching the Deliberative Wave (June 2020).
The report and this series focus on representative deliberative processes, which involve a randomly selected group of people, broadly representative of a community, taking time to learn and collaborate together through facilitated deliberation to form collective recommendations to policy makers, such as the Irish Citizens' Assembly.
In collaboration with our colleagues working on innovation in digital government and the public sector, we will draw on research by MySociety, NESTA and many other innovators and practitioners to analyse the state of the art in terms of digital support for deliberative processes.
Here we present the main issues that this series will explore:
1. How can digital tools support representative deliberative processes?
The current context forces policy makers and practitioners to think outside the box and adapt to the inability of physical deliberation. How can digital tools enable the continuation of planned or ongoing processes such as citizens' assemblies, ensuring that policymakers can still gather recommendations from informed citizens to inform their decision-making? New experiments are underway, and the evidence gathered could also be applied to other situations where face-to-face is not possible or more difficult, such as international processes or any situation that prevents physical gathering.
This series will cover the essential phases that a representative deliberative process should follow, as set out in the forthcoming OECD report: learning, deliberation, decision-making and collective recommendations. Because of the different nature of conducting an online process, we will also consider a necessary phase prior to learning: skills training. The papers will explore the use of digital tools in each phase, addressing issues of appropriate tools, methods, evidence and limitations.
They will also examine how the use of certain digital tools could reinforce good practice principles such as impact, transparency and evaluation:
Impact: Digital tools can help participants and the public to better monitor the progress of proposed recommendations and their impact on final decision-making. A parallel can be drawn with the extensive use of this methodology by the United Nations for monitoring and evaluating the impact of the Sustainable Development Goals (SDGs).
Transparency: Digital tools can facilitate transparency throughout the process. The use of collaborative tools allows for transparency regarding the authorship of the final result of the process (possibility to trace the contributors of the document and the different versions). By publishing the code and algorithms applied for the random selection (sorting) process and the data or statistics used for stratification, full transparency on how participants are selected could be achieved.
Evaluation: Data collection and analysis can help researchers and policy makers to evaluate the process (e.g. quality of deliberations, participant surveys, changes in opinion). Publishing these data in a structured and open format can allow for a broader evaluation and contribute to research. Over the next year, the OECD will prepare evaluation guidelines in line with the principles of good practice to enable comparative analysis of data.
The series will also look at how the use of emerging technologies and digital tools could complement face-to-face processes, for example:
Artificial intelligence (AI) and text-based technologies (i.e. natural language processing, NLP): Could the use of AI-based tools enrich deliberation processes? For example: mapping of opinion groups, consensus building, analysis of massive contributions from external participants at the early stage of stakeholder input. Could NLP enable simultaneous translation into other languages, sentiment analysis and automatic transcription? These possibilities already exist, but raise more relevant questions about reliability and user experience. How could they be linked to human analysis, discussion and decision making?
Virtual reality/augmented reality: Could the development of these emerging technologies allow participants to be immersed in virtual environments and thus simulate face-to-face deliberations and experiences that enable and enhance empathy with possible futures or experiences of others?
2. What are the limits of using digital tools for representative deliberative processes?
The use of digital tools in deliberative processes faces the same limitations as in many other participatory processes. The series will also explore and uncover limitations, such as :
Lack of social interaction: Online interaction may not have the same effect as face-to-face. However, there are interesting studies on the quality of socialisation in online spaces. The cases of online communities such as Anonymous or the social bonding during social movements (Occupy, Arab Spring, Sunflower Movement, Indignados) can help to gather evidence on online social interaction. A further comparison could be made with the online gaming community and the subsequent social interaction built through online and digital tools.
The digital divide: For digital tools to enable inclusive and equal participation among participants, we need to address the different types of exclusion (skills, access, gender, income, uses) and how to mitigate them.
Harmful technology: Technology is not neutral and can have negative effects on the democratic process (dependence on proprietary software, privacy issues, etc.). When using digital tools, organisers, policy makers and participants should be aware of the associated risks and give due consideration to the ethical implications of any use of technology or personal data.
Trust and legitimacy: What are the impacts on trust between participants, and also on public trust in virtual processes? What are the public perceptions of the legitimacy of online deliberative processes?
3. In what other contexts could these lessons be applied?
Although this series focuses primarily on how digital tools can improve representative deliberative processes, such as citizens' assemblies, juries and committees, there are many takeaways that could be useful in other situations.
The question can also be turned around. National parliaments, universities and political parties are using digital tools to adapt to 21st century methods and dynamics. This series will also include examples of practices of the above-mentioned institutions to adapt the main phases of a deliberative process based on their experience in using digital tools for deliberation and citizen participation.
The Digital for Deliberation series focuses on the use of technology, but the aim is to contribute to the general discussion on representative deliberative processes for policy making and, more broadly, to the implementation of the open government principle of citizen participation.
This article is a translation by Open Source Politics of the article published on the Medium "Participo", an OECD publication. To read the original article by Mauricio Mejia, click here.
The OECD's Innovative Citizen Participation workstream explores the ongoing paradigm shifts towards more inclusive governance. The idea is to better understand the new forms of deliberative, collaborative and participatory decision-making that are taking place, analysing what works well and what does not, and asking how democratic institutions might change in the long term as a result. (See our first article on Participo which explains the context).
The first report in this work stream focuses on the use of representative deliberative processes by public institutions. Catching the Deliberative Wave: Innovative Citizen Participation and New Democratic Institutions (June 2020), is the first international empirical comparative study to examine the functioning of representative deliberative processes for public decision-making and discusses the desirability of institutionalising them.
The report focuses on many aspects of deliberative processes, but recognises that this is the first step in more in-depth research on deliberation for public policy making. The use of digital tools to enrich deliberative processes is an area for further research and discussion.
The current situation and the consequences of COVID-19 will certainly have an impact on the way we interact and carry out our daily (and civic) activities. At the time of writing, millions of people are physically isolated and trying to carry out their normal activities. We are adapting to new tools and methods to make the most of working from home and maintaining social ties with friends, colleagues and family.
The coronavirus epidemic is also affecting the organisation of ongoing and future deliberative processes. The French and British Citizens' Assemblies on climate, for example, are considering the use of digital conferencing tools to replace or complement physical meetings that are postponed until further notice.
Even long-standing traditional institutions such as the European Parliament, the British House of Commons and the Lebanese Parliament are considering the use of video tools and electronic voting applications to replace face-to-face parliamentary debates and meetings. Other organisations, such as political parties, are also implementing digital tools to carry on their normal activities and could be a source of inspiration for our work.
Rather than replacing face-to-face deliberation with digital deliberation, we want to gather all relevant evidence on how digital tools and practices can enhance and support face-to-face deliberation processes. While the focus will be on the use of technology, the aim is to contribute to the general discussion and framework of deliberative processes for policy making. In other words, digital tools are the means to an end, not an end in themselves.