Co-Designing for Transparency: Lessons from Building a Document Organization Tool in the Criminal Justice Domain

Investigative journalists and public defenders conduct the essential work of examining, reporting, and arguing critical cases around police use-of-force and misconduct. In an ideal world, they would have access to well-organized records they can easily navigate and search. In reality, records can come as large, disorganized data dumps, increasing the burden on the already resource-constrained teams. In a cross-disciplinary research team of stakeholders and computer scientists, we worked closely with public defenders and investigative journalists in the United States to co-design an AI-augmented tool that addresses challenges in working with such data dumps. Our Document Organization Tool (DOT) is a Python library that has data cleaning, extraction, and organization features. Our collaborative design process gave us insights into the needs of under-resourced teams who work with large data dumps, such as how some domain experts became self-taught programmers to automate their tasks. To understand what type of programming paradigm could support our target users, we conducted a user study (n=18) comparing visual, programming-by-example, and traditional text-based programming tools. From our user study, we found that once users passed the initial learning stage, they could comfortably use all three paradigms. Our work offers insights for designers working with under-resourced teams who want to consolidate cutting-edge algorithms and AI techniques into unified, expressive tools. We argue user-centered tool design can contribute to the broader fight for accountability and transparency by supporting existing practitioners in their work in domains like criminal justice.