2021-02-11: Bruno Gavranović (MSP)

Sat, 21 Oct 2023 21:24:08 +0100

Bruno Gavranović (MSP)

Categorical Foundation of Gradient-Based Learning

Abstract

I will give an introduction to the categorical foundation of gradient-based learning algorithms. I'll define three abstract constructions and show how they can be put together to form general neural networks. The Para construction is used to compose neural networks while keeping track of their weights. Lenses/Optics which are used to take care of the forward-backward data flow and lastly, reverse derivative categories are used to functorially construct the backward wires from the forward ones. In addition, we'll see that gradient descent, Momentum, and a number of optimizers are lenses too, and that this framework includes learning on boolean circuits, in addition to standard Euclidean spaces.

2021-02-11 15:00:00 UTC
Online