You use (transfinite, if your ordinals are large enough) recursion! Just define a + 1 to be the successor of a — succ(a) — and then, assuming we’ve defined a + b, define
a + (b + 1) := (a + b) + 1 = succ(a + b)
(it’s only slightly more complicated for infinite ordinals)
You can do a similar thing for multiplication, and exponents, and so on.
Technically, you have to use induction to prove that this definition indeed works to define the operations for all ordinals.
For example, set union becomes the max function.