Multilabel Rule Learning

• Applicants: Professor Dr. Eyke Hüllermeier; Dr. Eneldo Loza Mencía, since 11/2019
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: from 2018 to 2022
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 400845550

• Final Report Year: 2021

Final Report Abstract

Inductive rule learning is one of the most traditional research areas in machine learning. Rule learning algorithms are typically used when not only the accuracy of predictions is of interest, but also the interpretability and transparency of the model, which ideally yields new insights into the domain of the dataset. In this project, we tackled the problem of learning interpretable rule models for the task of multi-label classification, which differs from conventional multi-class classification in that not only a single label but an entire subset of labels has to predicted. This problem is challenging, because dependencies between the labels have to be taken into account in order to obtain performant models. We formalized this problem in a formal way and clearly stated the associated challenges, such as whether individual rules should only predict single labels or multiple labels, whether they should make only positive or also negative predictions, or whether the rule body can contain predictions to other labels, which may result in circular reasoning structures. During the course of the project, we studied important subproblems, such as how multi-label losses for individual rules can be minimized, how it can be decided whether a rule should predict a label or abstain, how multiple predictions of individual rules can be aggregated into overall multi-label predictions, what strategies can be used to learn rule sets or decision lists, and how to deal with mixed dependency rule sets, which include both input features as well as label predictions of other rules in the rule bodies. These studies culminated in the development of BOOMER, a highly performant multi-label rule learning algorithm based on a gradient boosting methodology, which has previously been used in single-label rule learning. The learner is freely available and has already been adopted by other research groups.

Publications

• Learning interpretable rules for multi-label classification. In Jair Escalante, H. et al. (eds.), Explainable and Interpretable Models in Computer Vision and Machine Learning, The Springer Series on Challenges in Machine Learning, pp. 81–113. Springer-Verlag, 2018. ISBN 978-3-319-98131-4
  Loza Mencía, E., Furnkranz, J., Hullermeier, E., and Rapp, M.
  
• On the trade-off between consistency and coverage in multi-label rule learning heuristics. In Kralj Novak, P. et al.(eds.), Discovery Science, pp. 96–111, Cham, October 2019. Springer International Publishing. ISBN 978-3-030-33778-0
  Rapp, M., Loza Mencía, E., and Fürnkranz, J.
  
• Conformal rule-based multi-label classification. In Schmid, U., Klugl, F., and Wolter, D. (eds.), KI 2020: Advances in Artificial Intelligence. Springer, Cham, September 2020. ISBN 978-3-030-58284-5
  Hüllermeier, E., Fürnkranz, J., and Loza Mencía, E.
  
• Learning gradient boosted multi-label classification rules. In Proceedings of the European Conference of Machine Learning and Knowledge Discovery in Databases (ECML/PKDD), pp. 124–140. Springer, 2020
  Rapp, M., Loza Mencía, E., Fürnkranz, J., Nguyen, V.-L., and Hüllermeier, E.
  
• On aggregation in ensembles of multilabel classifiers. In Appice, A. et al. (eds.), Discovery Science, pp. 533–547, Cham, October 2020. Springer International Publishing. ISBN 978- 3-030-61527-7
  Nguyen, V.-L., Hüllermeier, E., Rapp, M., Loza Mencía, E., and Fürnkranz, J.
  
• Reliable multilabel classification: Prediction with partial abstention. Proceedings of the AAAI Conference on Artificial Intelligence, 34(04):5264–5271, Apr. 2020
  Nguyen, V.-L. and Hüllermeier, E.
  
• Rule-based multi-label classification: Challenges and opportunities. In Gutierrez-Basulto, V. et al. (eds.), Rules and Reasoning, pp. 3–19. Springer International Publishing, August 2020. ISBN 978-3-030-57977-7
  Hüllermeier, E., Fürnkranz, J., Loza Mencía, E., Nguyen, V.-L., and Rapp, M.
  
• Gradient-based label binning in multi-label classification. In Proceedings of the European Conference of Machine Learning and Knowledge Discovery in Databases (ECML/PKDD), Part III, pp. 462–477, 2021
  Rapp, M., Loza Mencía, E., Fürnkranz, J., and Hüllermeier, E.
  
• Multilabel classification with partial abstention: Bayes-optimal prediction under label independence. Journal of Artificial Intelligence Research, 72:613–665, 2021
  Nguyen, V.-L. and Hüllermeier, E.