Accountable Machine Learning Algorithms 2020
- Rescheduled to 28 June - 2 July 2021 -
25 - 29 May 2020
Venue: Lorentz Center@Snellius
If you are invited or already registered for this workshop, you have received login details by email.This workshop has been rescheduled to 28 June - 2 July 2021
Description
Intelligent automated systems are increasingly powerful and ubiquitous in all aspects of our lives, whether we are shopping online, being admitted to a hospital or, in some cases, being sentenced to a prison term. These systems have been facilitating human decision making, but they have also been raising concerns that they might be biased or may be generating highly sensitive outcomes for end-users.
In this workshop, we will focus on tangible approaches to tackle accountability, transparency, explainability and privacy challenges in designing, developing and using data-driven algorithms and automated decision-making systems.
Through this workshop, we aim to:
- Bring in practitioners’ point of view who have to work with real-world datasets,
- Understand practical challenges in both industry and academia; evaluate and remedy the limitations of existing solutions for accountable algorithms in a domain-specific way,
- Collaboratively design an inclusive research agenda with tangible research questions.
- Challenge SolveIt: Drafting solutions for the challenges raised by the case studies. The solutions could be existing tools, or well-formalized research questions that can potentially lead to the solutions needed.
- Challenge StirIt: Challenging the participants regarding the solutions they have been most familiar with, mixing the case studies across groups, and asking the groups to rate the solutions drafted by the other groups. The idea here is to realize the extent of generalisability of proposed solutions or the relevance of derived research questions across disciplines and fields. For example, can we extend the research questions raised in the earlier sessions across the different cases? Are the discussions related to one case study also relevant to the other ones?
- Challenge ServeIt: This is an ‘expert hot-tubbing' exercise. We borrow this way of 4 working from the American judicial system where expert witnesses from opposing sides are asked questions concurrently. The participants are encouraged to come to a compromise, within a reasonable time frame, for the benefit of the case under scrutiny. We use this method to encourage productive opinion exchange between participants who do not agree on important topics, while removing the confrontational aspect of the exchange.
Monday May 25
Introduction/Setting the scene
09.00 – 10.00 Arrival, registration with coffee & tea
10.00 – 10.15 Welcome by the Lorentz Center
10.15 – 11.00 Opening by the scientific organizers: welcome, explanation of workshop objectives and way of working during the week
11.00 – 12.00 Round of speedy introductions: all participants
12.00 – 14.00 Lunch @Snellius restaurant
14.00 – 15.00 Pitches of use-cases from selected participants
15.00 – 16.00 Coffee & tea break (Poster/Demo session)
16.00 – 17.00 Pitches of use-cases - continued
17.00 – 17.30 Wrap-up: identification of the common interests and formation of the discussion groups
17.30 – Wine & cheese party (Poster/Demo session)
Tuesday May 26
Focus: XAI (explainability, interpretability)
09:00 – 09:30 Introduction to Day 2 by chair
09.30 – 10.30 Tutorial on XAI: status-quo, challenges, prospects, experiences
10.30 – 10.40 Change of rooms
10.40 – 12:00 Group discussions in separate rooms - SolveIT1 (with coffee and tea)
12.00 – 13.30 Lunch and informal discussions
13.30 – 15.00 Group discussions in separate rooms - StirIT
15.00 - 15.30 Coffee & tea break (Poster/Demo session)
15:30 - 16:30 Group discussions in main room - ServeIT (all participants)
16.30 – 17.30 Panel with group reporters & closing discussion of the day
Wednesday May 27
Focus: Transparency & Privacy
09:00 – 09:30 Introduction to Day 3 by chair
09.30 – 10.30 Tutorial on transparency and privacy: status-quo, challenges, prospects, experiences
10.30 – 10.40 Change of rooms
10.40 – 12:00 Group discussions in separate rooms - SolveIT (with coffee and tea)
12.00 – 13.30 Lunch and informal discussions
13.30 – 15.00 Group discussions in separate rooms - StirIT
15.00 – 15.30 Coffee & tea break (Poster/Demo session)
15:30 – 16:30 Group discussions in main room - ServeIT (all participants)
16.30 – 17.30 Panel with group reporters & closing discussion of the day
17.30 – 22:00 Departure for activity and dinner
Thursday May 28
Focus: Accountability & Fairness
9:00 – 9:30 Introduction to Day 4 by chair
9.30 – 10.30 Tutorial on accountability and fairness: status-quo, challenges, prospects, experiences
10.30 – 10.40 Change of rooms
10.40 – 12:00 Group discussions in separate rooms - SolveIT (with coffee and tea)
12.00 – 13.30 Lunch and informal discussions
13.30 – 15.00 Group discussions in separate rooms - StirIT
15.00 - 15.30 Coffee & tea break (Poster/Demo session)
15:30 - 16:30 Group discussions in main room - ServeIT (all participants)
16.30 – 17.30 Panel with group reporters & closing discussion of the day
Friday May 29
Wrap up, summarize and inspire
10.00 - 11.30 Looking back and forward: Wrap-up of the workshop, (action) plans for the future, outline what practitioners can take with them
11.30 - 12.00 Lessons learned from the community (Amsterdam Data Science)
12:00 - Lunch and goodbye
Way of working
Throughout discussions, we will use the following strategies:
‘Hot-tubbing’ is the name for the (US) court process of calling expert witnesses to give evidence and be
cross-examined at the same time. It encourages opposing experts to engage in constructive discussions in the witness box (in front of judges/juries)
