Fix Mistake for Adjoint of ODEs

[aleppersico]

Suggestion

There appears to be an error in lecture 11 "Differentiable Programming and Neural Differential Equations", subsection "Adjoint of Ordinary Differential Equations", specifically in the passage after integration by parts is applied in the derivation of $dG/dp$. The resulting expression seems to have incorrect signs.

Identification

• Section: Derivation of Reverse Mode Rules: Adjoints and Implicit Function Theorem
• Subsection: Adjoint of Ordinary Differential Equations

Correction

The derivation should make the post-integration-by-parts expression for ( dG/dp ) as follows:

$ \frac{dG}{dp}
= \int_{t_0}^{T}{(g_p + g_u s) dt}

• | \lambda^*(t) s(t)|_{t_0}^{T}

• \int_{t_0}^{T}{\lambda^{*}' s dt}
• \int_{t_0}^{T}{\lambda^{} (f_u s + f_p) dt}
  = \int_{t_0}^{T}{(g_p + \lambda^ f_p) dt}

• | \lambda^*(t) s(t)|_{t_0}^{T}

• \int_{t_0}^{T}{(\lambda^' + \lambda^ f_u + g_u) s dt}$

(current version have the signs inverted).

That was just a re-arrangement. Now, let's require that
$\lambda'
= - {\frac{df}{du}}^* \lambda

• \left(\frac{dg}{du}\right)^*$

$ \lambda(T) = 0$

This means that one of the boundary terms of the integration by parts is zero, and also one of those integrals is perfectly zero. Etc.

Screenshots

Not included here.

Additional context

The issue seems to be in the algebra immediately after integration by parts. The final adjoint construction becomes clear once the signs are written consistently.