State-Space Model

A State-Space Model is a mathematical model that represents a system in terms of a set of input variables, output variables, and state variables (to represent the system's current status), all related by differential equations or difference equations.

• Context:
  • It can be expanded beyond Linear Systems and can incorporate non-linear dynamics, making it versatile for various applications.
  • ...
• Example(s):
  • Structured state space sequence models (S4)
  • In linear, time-invariant, and finite-dimensional systems, be expressed in matrix form.
  • In Control Engineering, designing a feedback controller for a robotic arm using state-space representation to model the arm's dynamics and control its movement accurately.
  • In Econometrics, applying a state-space model to decompose a Time Series into its trend and cycle components, aiding in economic forecasting and analysis.
  • In Neuroscience, modeling the dynamics of a neural network using state-space models to understand how neural states evolve over time in response to stimuli.
  • In Electrical Engineering, using state-space models for the design and analysis of complex electrical circuits and systems, where the state variables represent voltages and currents.
  • ...
• Counter-Example(s):
  • a Frequency Domain approach.
• See: Time Series, Kalman Filter, Control Engineering, System Identification, Mathematical Model, Variable (Mathematics), Differential Equation, Difference Equation, Phase Space, Geometric Space, [[Vector (Mathematics)]

---

---

References

2024

• (Wikipedia, 2024) ⇒ https://en.wikipedia.org/wiki/State-space_representation Retrieved:2024-1-5.
  • In control engineering and system identification, a state-space representation is a mathematical model of a physical system specified as a set of input, output and variables related by first-order (not involving second derivatives) differential equations or difference equations. Such variables, called state variables, evolve over time in a way that depends on the values they have at any given instant and on the externally imposed values of input variables. Output variables’ values depend on the values of the state variables and may also depend on the values of the input variables.

    The state space or phase space is the geometric space in which the variables on the axes are the state variables. The state of the system can be represented as a vector, the state vector, within state space.

    If the dynamical system is linear, time-invariant, and finite-dimensional, then the differential and algebraic equations may be written in matrix form. The state-space method is characterized by significant algebraization of general system theory, which makes it possible to use Kronecker vector-matrix structures. The capacity of these structures can be efficiently applied to research systems with modulation or without it. The state-space representation (also known as the "time-domain approach") provides a convenient and compact way to model and analyze systems with multiple inputs and outputs. With [math]\displaystyle{ p }[/math] inputs and [math]\displaystyle{ q }[/math] outputs, we would otherwise have to write down [math]\displaystyle{ q \times p }[/math] Laplace transforms to encode all the information about a system. Unlike the frequency domain approach, the use of the state-space representation is not limited to systems with linear components and zero initial conditions. The state-space model can be applied in subjects such as economics, statistics, computer science and electrical engineering, and neuroscience. In econometrics, for example, state-space models can be used to decompose a time series into trend and cycle, compose individual indicators into a composite index, identify turning points of the business cycle, and estimate GDP using latent and unobserved time series. Many applications rely on the Kalman Filter or a state observer to produce estimates of the current unknown state variables using their previous observations.

2023

• (Gu & Dao, 2023) ⇒ Albert Gu, and Tri Dao. (2023). “Mamba: Linear-Time Sequence Modeling with Selective State Spaces.” doi:10.48550/arXiv.2312.00752
  • QUOTE: ... Structured state space sequence models (S4) are a recent class of sequence models for deep learning that are broadly related to RNNs, and CNNs, and classical state space models. They are inspired by a particular continuous system (1) that maps a 1-dimensional function or sequence x(t) E IR � y(t) E IR through an implicit latent state ...

2021

• (Gu et al., 2021) ⇒ Albert Gu, Karan Goel, and Christopher Ré. (2021). “Efficiently Modeling Long Sequences with Structured State Spaces.” In: arXiv preprint arXiv:2111.00396.
  • NOTE: This paper addresses the challenge of sequence modeling, especially for long-range dependencies in sequences exceeding 10,000 steps. While RNNs, CNNs, and Transformers offer specialized variants for long dependencies, they face scalability issues. The paper discusses the use of the state space model (SSM) \( x'(t) = Ax(t) + Bu(t), y(t) = Cx(t) + Du(t) \) and proposes a new model, the Structured State Space sequence model (S4). The S4 model, with its novel parameterization of the SSM, is computationally efficient and theoretically robust, effectively handling long-range dependencies. It applies a low-rank correction to the state matrix \( A \), leading to a stable diagonalization and reducing the SSM to a computation involving a Cauchy kernel. The model demonstrates strong empirical results across various benchmarks, including high accuracy in sequential CIFAR-10, closing the gap with Transformers in image and language modeling tasks, and achieving state-of-the-art results in the Long Range Arena benchmark. Notably, it solves the Path-X task of length 16k, where prior models struggled, while maintaining efficiency.

2013

• (Chen & Brown, 2013) ⇒ Z. Chen, and E.N. Brown. (2013). “State-Space Models.” In: Scholarpedia.
  • QUOTE: State space model (SSM) refers to a class of probabilistic graphical model (Koller and Friedman, 2009) that describes the probabilistic dependence between the latent state variable and the observed measurement. The state or the measurement can be either continuous or discrete. The term “state space” originated in 1960s in the area of control engineering (Kalman, 1960). SSM provides a general framework for analyzing deterministic and stochastic dynamical systems that are measured or observed through a stochastic process. The SSM framework has been successfully applied in engineering, statistics, computer science and economics to solve a broad range of dynamical systems problems. Other terms used to describe SSMs are hidden Markov models (HMMs) (Rabiner, 1989) and latent process models. The most well studied SSM is the Kalman filter, which defines an optimal algorithm for inferring linear Gaussian systems.

1998

• (Kitagawa, 1998) ⇒ G. Kitagawa. (1998). “A Self-Organizing State-Space Model.” In: Journal of the American Statistical Association, JSTOR.
  • NOTE: This paper discusses the limitations of traditional state-space models in handling certain problems and introduces several types of nonlinear non-Gaussian state-space models, including the dynamic generalized linear model.

1994

• (Hamilton, 1994) ⇒ J.D. Hamilton. (1994). “State-Space Models.” In: Handbook of Econometrics. Elsevier.
  • NOTE: This chapter reviews models of changes in regime and develops the parallel between such models and linear state-space models.

---