Merge pull request #152 from acfr/paper-revisions

Revisions

Author: nic-barbara

paper/TexFiles/Conclusions.tex

@@ -1,4 +1,4 @@
-This paper has presented \verb|RobustNeuralNetworks.jl|, a Julia package for robust machine learning based on the recently-proposed Recurrent Equilibrium Network (REN) and Lipschitz-Bounded Deep Network (LBDN) model classes. The models are unique in that they naturally satisfy a set of \textit{built-in} robustness certificates, such as contraction and Lipschitz bounds. We have presented an overview of the model architectures, including background theory on robustness metrics in nonlinear systems, and have outlined the package structure and its usage alongside Julia's main machine-learning library, \verb|Flux.jl|. We have demonstrated via examples in image classification, reinforcement learning, and observer design that the package is easy to use in many common machine learning and data-driven control problems, while also offering the advantage of robustness guarantees.
+This paper has presented \verb|RobustNeuralNetworks.jl|, a Julia package for robust machine learning based on the recently-proposed Recurrent Equilibrium Network (REN) and Lipschitz-Bounded Deep Network (LBDN) model classes. The models are unique in that they naturally satisfy a set of \textit{built-in} robustness metrics, such as contraction and Lipschitz bounds. We have presented an overview of the model architectures, including background theory on robustness metrics in nonlinear systems, and have outlined the package structure and its usage alongside Julia's main machine-learning library, \verb|Flux.jl|. We have demonstrated via examples in image classification, reinforcement learning, and observer design that the package is easy to use in many common machine learning and data-driven control problems, while also offering the advantage of robustness guarantees.
 
 We intend \verb|RobustNeuralNetworks.jl| to be widely-applicable in the scientific and machine learning communities for learning-based problems in which robustness certificates are crucial, and have already used the package in our own research in robust reinforcement learning \cite{Barbara++2023}. Some areas in which this package will be most applicable include: data-driven control and state estimation, image classification and segmentation, and privacy and security. We intend to expand the package with more robust neural network architectures in the future. Examples include LBDNs with one-dimensional convolution \cite{Pauli++2022c} and circular convolutions \cite{Wang+Manchester2023}, continuous-time REN models \cite{Martinelli++2023}, and RENs respecting other non-Euclidean contraction metrics \cite{Davydov++2022}. We encourage any and all contributions to \verb|RobustNeuralNetworks.jl| to further its use in robust machine learning problems.
 

paper/TexFiles/ExampleRL.tex

@@ -1,6 +1,6 @@
 \subsection{Reinforcement learning} \label{sec:rl}
 
-One of the original motivations for developing the model structures in  \verb|RobustNeuralNetworks.jl| was to guarantee stability and robustness in learning-based control. Recently, we have shown that with a controller architecture based on a nonlinear version of classical Youla-Kucera parameterization \cite{Youla++1976}, one can learn over a space of stabilizing controllers for linear and nonlinear systems using standard reinforcement learning techniques, so long as the control policy is parameterized by a contracting, Lipschitz-bounded REN  \cite{Wang+Manchester2022,Wang++2022,Barbara++2023}. This is an exciting result for learning-based controllers in safety-critical systems, such as in robotics.
+One of the original motivations for developing the model structures in  \verb|RobustNeuralNetworks.jl| was to guarantee stability and robustness in learning-based control. Recently, we have shown that with a controller architecture based on a nonlinear version of classical Youla-Ku\v{c}era parameterization \cite{Kucera1975,Youla++1976}, one can learn over a space of stabilizing controllers for linear and nonlinear systems using standard reinforcement learning techniques, so long as the control policy is parameterized by a contracting, Lipschitz-bounded REN  \cite{Wang+Manchester2022,Wang++2022,Barbara++2023}. This is an exciting result for learning-based controllers in safety-critical systems, such as in robotics.
 
 In this example, we will demonstrate how to train an LBDN controller with \textit{reinforcement learning} (RL) for a simple nonlinear dynamical system. This controller will not have any stability guarantees. The purpose of this example is simply to showcase the steps required to set up RL experiments for more complex systems with RENs and LBDNs.
 
@@ -111,7 +111,7 @@ \subsubsection{Problem setup} \label{sec:rl-setup}
 % Define a model
 \subsubsection{Define a model} \label{sec:rl-model}
 
-We will train an LBDN controller with a Lipschitz bound of $\gamma = 20$. Its inputs are the state $x_t$ and goal position $q_\mathrm{ref}$, while its outputs are the control force $u_t$. We have chosen a model with two hidden layers each of 32 neurons just as an example. For examples of how Lipschitz bounds can be useful in learning robust controllers, see \cite{Barbara++2023,Russo+Proutiere2021,Song++2023}.
+We will train an LBDN controller with a Lipschitz bound of $\gamma = 20$. Its inputs are the state $x_t$ and goal position $q_\mathrm{ref}$, while its outputs are the control force $u_t$. We have chosen a model with two hidden layers each of 32 neurons just as an example. For examples of how Lipschitz bounds can be useful in learning robust controllers, see \cite{Barbara++2024a,Russo+Proutiere2021}.
 
 \begin{lstlisting}[language = Julia]
 using RobustNeuralNetworks

paper/TexFiles/Introduction.tex

@@ -1,9 +1,9 @@
 Modern machine learning relies heavily on rapidly training and evaluating neural networks in problems ranging from image classification \cite{He++2016} to robotic control \cite{Siekmann++2021a}. Most neural network architectures have no robustness certificates, and can be sensitive to adversarial attacks and other input perturbations \cite{Huang++2017}. Many approaches that address this brittle behaviour rely on explicitly enforcing constraints during training to smooth or stabilize the network response \cite{Pauli++2022,Junnarkar++2023}. While effective on small-scale problems, these methods are computationally expensive, making them slow and difficult to scale up to complex real-world problems.
 
-Recently, we proposed the \textit{Recurrent Equilibrium Network} (REN) \cite{Revay++2023} and \textit{Lipschitz-Bounded Deep Network} (LBDN) or \textit{sandwich layer} \cite{Wang+Manchester2023} model classes as computationally efficient solutions to these problems. The REN architecture is flexible in that it includes all common neural network models, such as multi-layer-perceptrons (MLPs), convolutional neural networks (CNNs), and recurrent neural networks (RNNs). The weights and biases in RENs are directly parameterized to \textit{naturally satisfy} behavioural constraints chosen by the user. For example, we can build a REN with a given Lipschitz constant to ensure its output is quantifiably less sensitive to input perturbations. LBDNs are specializations of RENs with the specific feed-forward structure of deep neural networks like MLPs or CNNs and built-in guarantees on the Lipschitz bound.
+Recently, we proposed the \textit{Recurrent Equilibrium Network} (REN) \cite{Revay++2023} and \textit{Lipschitz-Bounded Deep Network} (LBDN) or \textit{sandwich layer} \cite{Wang+Manchester2023} model classes as computationally efficient solutions to these problems. RENs are flexible in that they include many common neural network models, such as multi-layer-perceptrons (MLPs), convolutional neural networks (CNNs), and recurrent neural networks (RNNs). Their weights and biases are parameterized to naturally satisfy a set of user-defined robustness metrics constraining the internal stability and input-output sensitivity of the network. When a network is guaranteed to satisfy a robust metric, we call this a \textit{robustness certificate}. An example is a Lipschitz bound, which restricts the network's amplification of input perturbations in its outputs \cite{Pauli++2022}. LBDNs are specializations of RENs with the specific feed-forward structure of deep neural networks like MLPs or CNNs, and built-in restrictions on the Lipschitz bound.
 
-The direct parameterization of RENs and LBDNs means that we can train models with standard, unconstrained optimization methods (such as stochastic gradient descent) while also guaranteeing their robustness. Achieving the “best of both worlds” in this way is the main advantage of the REN and LBDN model classes, and allows the user to freely train robust models for many common machine learning problems, as well as for more challenging real-world applications where safety is critical.
+This special parameterization of RENs and LBDNs means that we can train models with standard, unconstrained optimization methods (such as stochastic gradient descent) while also guaranteeing their robustness. Achieving the “best of both worlds” in this way is the main advantage of the REN and LBDN model classes, and allows the user to freely train robust models for many common machine learning problems, as well as for more challenging real-world applications where safety is critical.
 
-This papers presents \verb|RobustNeuralNetworks.jl|: a package for neural network models that naturally satisfy robustness constraints. The package contains implementations of the REN and LBDN model classes introduced in \cite{Revay++2023} and \cite{Wang+Manchester2023}, respectively, and relies heavily on key features of the Julia language \cite{Bezanson++2017} (such as multiple dispatch) for an efficient implementation of these models. The purpose of \verb|RobustNeuralNetworks.jl| is to make our recent research in robust machine learning easily accessible to users in the scientific and machine learning communities. With this in mind, we have designed the package to interface directly with \verb|Flux.jl| \cite{Innes2018}, Julia's most widely-used machine learning package, making it straightforward to incorporate our robust neural networks into existing Julia code.
+This papers presents \verb|RobustNeuralNetworks.jl|: a package for neural networks with built-in robustness certificates. The package contains implementations of the REN and LBDN model classes introduced in \cite{Revay++2023} and \cite{Wang+Manchester2023}, respectively, and relies heavily on key features of the Julia language \cite{Bezanson++2017} (such as multiple dispatch) for an efficient implementation of these models. The purpose of \verb|RobustNeuralNetworks.jl| is to make our recent research in robust machine learning easily accessible to users in the scientific and machine learning communities. We have therefore designed the package to interface directly with \verb|Flux.jl| \cite{Innes2018}, Julia's most widely-used machine learning package, making it straightforward to incorporate our robust neural networks into existing Julia code.
 
 The paper is structured as follows. Section \ref{sec:overview} provides an overview of the \verb|RobustNeuralNetworks.jl| package, including a brief introduction to the model classes (Sec. \ref{sec:model-structures}), their robustness certificates (Sec. \ref{sec:robustness}), and their implementation (Sec. \ref{sec:parameterizations}). Section \ref{sec:examples} guides the reader through a tutorial with three examples to demonstrate the use of RENs and LBDNs in machine learning: image classification (Sec. \ref{sec:mnist}), reinforcement learning (Sec. \ref{sec:rl}), and nonlinear state-observer design (Sec. \ref{sec:observer}). Section \ref{sec:conc} offers some concluding remarks and future directions for robust machine learning with \verb|RobustNeuralNetworks.jl|. For more detail on the theory behind RENs and LBDNs, and for examples comparing their performance to current state-of-the-art methods on a range of problems, we refer the reader to \cite{Revay++2023} and \cite{Wang+Manchester2023} (respectively).