Full Fact aim to put the first fully automated fact-checking system into use within the next year. To support this work, Mevan Babakar and Will Moy published The State of Automated Fact-Checking, which calls for increased collaboration and the adoption of international fact-checking standards. While acknowledging that more research is necessary to deliver reliable speech recognition and claims identification, the authors argue that that "end-to-end" fact-checking is already possible using existing technology.
Fact-checking is the same four stage process whether it's done by humans or machines. Technology which integrates these four stages, achieving automated "end-to-end" fact-checking, is well within our sights.
International Collaboration: Although fact-checking has grown worldwide, projects by news outlets, independent organisations and researchers remain disconnected from each other. The lack of coordination means that fact-checkers often waste time and money duplicating work. Moreover, Full Fact argue that increased collaboration is vital if improved automation is to be achieved. To this end, the authors recommend five principles for "shared international standards and systems for building automated fact-checking tools".
We propose five principles:
- Standard data formats, so that any new automated fact-checking tool can work with any known source.
- Shared monitoring systems, so we do not duplicate work unnecessarily.
- Open and shared evaluation, so we know what works and what it works for.
- Published roadmaps, to attract volunteers, researchers, partners and funders to work with us.
- Think global, so that where possible new automated fact-checking tools are designed with the aim of being able to work for many languages and countries.
Full Fact's Roadmap: Full Fact's system has four components which monitor, spot statistical claims, check them, prepare responses, and publish without human intervention. The system is not yet complete for internal use but the aim is to make it available externally by the end of year.
Hawk, which is our monitoring and claim recognition engine. It monitors public debate and spots claims we have previously factchecked. It is in internal beta.
Stats, which is our tool for automatically checking statistical claims. It helps us fact-check more quickly and efficiently. It is a proof of concept, moving into prototype stage.
Trends, which is our monitoring product. It shows how common a claim is, where it is being made, and who is making it, using the results from Hawk. It helps us scale, target, and evaluate our work. It is in internal beta.
Robocheck, which is our real time product. It will provide subtitles of live TV, and add verdicts to claims in real time using the results from Stats and Hawk. It will help journalists and others hold public figures to account. It is in design phase.
Design Principles for Factchecking Tools: To build a system that is flexible and robust, Full Fact endorse the following design principles:
- Make simple tools that work and make fact-checking more effective, then make them work better;
- Make each simple tool do one thing well, and work well with others;
- Make tools that will scale;
- Design tools to be adaptable to languages other than English;
- Focus on taking existing technology and applying it to making factchecking more effective, not on pushing the boundaries of computer science research.
The report includes an overview of automated fact-checking projects and the authors invite participation in the development of open, collaborative, and standards-based tools for automated fact-checking.
People are attacking the problem from two sides: factcheckers trying to get practical tools into actual use, and computer scientists trying to solve deep underlying research problems.
Read the full report (pdf).
Subscribe to Fujo's newsletter
