Visual Comparison of Classifier Models

Theoretical (Analytical):

Practical (Implementation):

Literature Work:


Problem Statement

Recent advances in Machine-Learning drew a lot of attention to this area in Computer Science. Classification is one sub-area in Machine-Learning, and there are several examples of classification problems like image recognition and prediction of diseases in the health sector.

However, there are a lot of classification models around, and the comparison of its results to select the most suitable model is not a trivial task. Visualization has been playing a role in this area, but there is still room for improvement.

In this topic proposal, we aim to develop novel visualization and interaction techniques for classifier model comparison, guided by the following questions: Which features of the data (classification outputs) and the models should we visualize for effective model comparison? How can interaction support this task? How to navigate through distinct levels of aggregation in the classification outputs, to search for hidden patterns in this data? Does the visualization of individual representative predictions can help in the comparison?

For Master Students, we plan a user study to evaluate and refine the prototype(s).

Tasks

  • Get familiar with classification models and its workflow.

  • Survey and categorize existing visual methods for classifier model comparison.
  • Experiment with different visualizations for classifier model comparison, and implement a first interactive model comparison prototype.
  • Set up a simple user-study with our support to evaluate the prototype (only for Master students).

Requirements

  • Good knowledge in Java or HTML/JavaScript programming languages

  • Basic knowledge or the interest in learning about machine-learning classifiers

Scope/Duration/Start

  •  Scope: Bachelor/Master
  • 6 Month Project, 3 Month Thesis (Bachelor) / 6 Month Thesis (Master)
  • Start: immediately

Contact

References

  • M.T. Ribeiro, S. Singh, and C. Guestrin. Why Should I Trust You?: Explaining the Predictions of Any Classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 2016.
  • B. Alsallakh, A. Hanbury, H. Hauser, S. Miksch, and A. Rauber. Visual methods for analyzing probabilistic classification data. IEEE Transactions on Visualization and Computer Graphics, 20(12):1703–1712, 2014.
  • D. Ren, S. Amershi, B. Lee, J. Suh, and J. D. Williams. Squares: Supporting interactive performance analysis for multiclass classifiers. IEEE Transactions on Visualization and Computer Graphics, 23(1):61–70, 2017