Project description
Tsetlin Machines (TMs) are a new machine learning (ML) approach founded on the Tsetlin Automaton. TMs use frequent pattern mining and resource allocation to extract common patterns in the data, rather than relying on minimizing output error, which is prone to overfitting. Unlike the intertwined nature of pattern representation in neural networks (NNs), TMs decompose problems into self-contained patterns, each represented as a conjunctive clause. The clause outputs, in turn, are combined into a classification decision through summation and thresholding, akin to a logistic regression function, however, with binary weights and a unit step output function. TM hardware (HW) has demonstrated up to three orders of magnitude reduced energy and faster learning, compared to NNs alike. Logic-driven fundamental blocks, organized in lean parallel processing units, are major contributors to this comparative advantage over NNs that are heavily arithmetic-based. The TM further outperforms vanilla CNNs and LSTMs accuracy-wise on well-established benchmarks. While the reported results on TMs are promising in terms of scalability, training time, accuracy, and energy, we here address three major obstacles. 1) Current FPGA and ASIC prototypes lack scalable memory elements, constraining them to small-scale ML problems. 2) Reinforcement learning (RL) is key to many ML problems, such as playing board games, however, it is unclear how to model RL in the TM framework. 3) State-of-the-art deep learning models support pre-training on unlabelled data, which significantly improves the accuracy of following supervised learning, dealing with shortage of labelled data. It is unclear how to pre-train TMs from unlabelled data. By overcoming these three obstacles we aim to architect a new TM HW/SW ecosystem that outperforms state-of-the-art ML in terms of energy efficiency and scalability, parametrised by accuracy. This will enable powerful logic-based ML applications at the edge and in the cloud.
