In the video they present it as an algebraic framework where you can "add" to a number system elements x such that x^2 = -1, x^2 = 0, or x^2 = 1. Adding the element x^2 = -1 to the real numbers gives you the complex numbers, with x = i. Adding the element x^2 = 0 to the real numbers gives you the dual numbers with x = epsilon, which is what can be used for automatic differentiation. The case of x^2 = 1 is more complicated.
Newer ML frameworks do source to source transformations, which allows calculating the derivative without changing the function signature, but the concepts used remain the same.
Could you extend more on how GA plays a role in this? I wish to learn more but don't know where to start. Any leads would be appreciated greatly.