Lorentz Center - Lorentz Center Highlights
  Current Workshop  |   Overview   Back  |   Print   |   Home   |   Search   |     

    Lorentz Center Highlights

Competitions a healthy way to improve verification technology

Fabrice Kordon University of Paris-Sorbonne
Martina Seidl Johannes Kepler University of Linz
Jaco van de Pol Aarhus University + University of Twente

ABSTRACT: Software and hardware verification is a complex task, supported by automated reasoning tools. Tremendous progress has led to a multitude of algorithms and tools that provide automated and scalable solutions for specific verification instances. The research field has been organizing several series of competitions as means for the objective evaluation and comparison between verification tools on a common set of benchmarks. These competitions provide insight in the best solutions for a particular task, but they also motivate researchers to push the boundaries of their tools, improving the state-of-the-art. This is why notable events have had significant impact on the involved communities. What are the success factors for such competitions? And how do we optimize the learning outcome of verification competitions?
We discuss:
how to setup the rules and to select benchmarks of a competition
how to execute the competition itself
how to evaluate and communicate the end results