Inductive Transfer with Deep Biases

• Applicant: Dr. Ulrich Rückert
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: from 2009 to 2012
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 137584527

In order to induce models with high predictive accuracy, machine learning methods must either have access to a sufficiently large amount of training data or feature an inductive bias that matches particularly well with the data distribution at hand. If there is only a limited amount of training data available, it is thus sensible to adjust the inductive bias of a learning system to better match the current task. One way to do so is inductive transfer. Here, the main idea is to transfer inductive biases that have been successfully used in the past on similar or related learning tasks to the new learning task at hand. In this project, we address the question of how inductive transfer can be efficiently implemented for learning systems with “deep” biases, that is, systems that can learn complex non-linear properties from structured training data. To do so, we investigate theoretically and empirically how deep biases can be represented, identified, compared, merged and transferred. We aim specifically at efficient algorithms that are not application specific, but perform well under broad conditions. The implementations of such algorithms are evaluated on three sample applications, namely structure-activity-relationships with molecular graph data, text document classification and machine translation.

• DFG Programme: Research Fellowships
• International Connection: USA
• Host: Professor Michael I. Jordan