Data assimilation is a crucial component in the Earth science field, enabling the integration of observation data with numerical models. In the context of numerical weather prediction (NWP), data assimilation is particularly vital for improving initial conditions and subsequent predictions. However, the computational demands imposed by conventional approaches, which employ iterative processes to minimize cost functions, pose notable challenges in computational time. The emergence of quantum computing provides promising opportunities to address these computation challenges by harnessing the inherent parallelism and optimization capabilities of quantum annealing machines.

In this investigation, we propose a novel approach termed quantum data assimilation, which solves the data assimilation problem using quantum annealers. Our data assimilation experiments using the 40-variable Lorenz model were highly promising, showing that the quantum annealers produced an analysis with comparable accuracy to conventional data assimilation approaches. In particular, the D-Wave Systems physical quantum annealing machine achieved a significant reduction in execution time.

Data assimilation is a mathematical discipline that integrates numerical models and observation data to improve the interpretation and predictions of dynamical systems (Reichle, 2008; Evensen, 2009). In particular, data assimilation has been intensively investigated in numerical weather prediction (NWP) during the past 2 decades to provide optimal initial conditions by combining model forecasts and observation data (Kalnay, 2003; Houtekamer and Zhang, 2016). Among data assimilation methods, variational and ensemble–variational data assimilation methods, which iteratively reduce cost functions via gradient-based optimization, are widely used in most operational NWP centers such as the European Centre for Medium-Range Weather Forecasts (ECMWF), the United Kingdom Met Office (Met Office), the National Oceanic, Atmospheric Administration of the United States (NOAA) and the Japan Meteorological Agency (JMA). However, data assimilation methods require vast computational resources in NWP systems because of the iterations needed for sufficient cost function reduction. For example, in JMA's global forecast system, data assimilation requires about 25 times more computational resources than forecast computations.

In recent years, quantum computing has attracted research interest as a new paradigm of computational technologies since it has a large potential to overcome computational challenges of conventional approaches through quantum effects such as tunneling, superposition and entanglement. In particular, quantum annealing machines (Kadowaki and Nishimori, 1998), such as D-Wave Systems quantum annealers (Johnson et al., 2011), are powerful and feasible tools for solving optimization problems. Since the quantum annealer 2000Q was released by D-Wave Systems in 2017, quantum computing research has rapidly progressed in various applications, such as for machine learning (Hu et al., 2019; Willsch et al., 2020), graph partitioning (Ushijima-Mwesigwa et al., 2017), clustering (O'Malley et al., 2018), and model predictive control (Inoue et al., 2021).

In this study, we design a data assimilation method for the quantum annealing machines. Although quantum machines have been used in several engineering applications, to our knowledge, this is the first study to apply quantum annealing to data assimilation problems. We focus on the four-dimensional variational data assimilation (4DVAR) since it is the most widely used data assimilation method in operational NWP systems. We reformulate 4DVAR into a quadratic unconstrained binary optimization (QUBO) problem, which can be solved by quantum annealers. We subsequently apply the proposed method for a series of 4DVAR experiments using a low-dimensional chaotic Lorenz 96 model (Lorenz, 1996; Lorenz and Emanuel, 1998), which has been widely used in theoretical data assimilation studies (e.g., Anderson, 2011; Whitaker and Hamill, 2002; Miyoshi, 2011; Kotsuki et al., 2017).

The original 4DVAR cost function is, as elaborated in Sect. 2.1, a quadratic unconstrained optimization (QUO) problem including a nonlinear operator with respect to the analysis increment (NL-QUO). To solve 4DVAR using quantum annealers, we first approximate the problem so as to include only linear operations with respect to the analysis (L-QUO), which is then reformulated to a be quadratic unconstrained binary optimization (L-QUBO) problem. The L-QUBO problem is solved using the D-Wave Advantage physical quantum annealer (Phy-QA) and the Fixstars Amplify simulated quantum annealer (Sim-QA). We also employ the conventionally used quasi-Newton method with the Broyden–Fletcher–Goldfarb–Shanno formula (BFGS) to solve the NL-QUO and L-QUO, which are denoted as NL-BFGS and L-BFGS hereafter. Numerical techniques and practical implementations specifically tailored to quantum data assimilation are also presented.

The rest of paper is organized as follows. Section 2 provides the method of quantum data assimilation, and Sect. 3 provides results and discussion. Finally, a summary is presented in Sect. 4.

Conceptual image of 4DVAR data assimilation. Green and red lines indicate the trajectories of the first guess and updated analysis, respectively. Blue circles with error bars represent observations.

This study focuses on the four-dimensional variational data assimilation (4DVAR), which is among the most widely used data assimilation methods in operational NWP centers such as ECMWF, Met Office, NOAA, and JMA. 4DVAR assimilates observations over a time window to produce an analysis trajectory that minimizes its cost function (Fig. 1). The cost function of 4DVAR is derived from Bayes' theorem and is given by

As an intermediate step toward QUBO, the original cost function is approximated as follows:

Quantum annealers require only the cost function in contrast to conventional 4DVAR that requires the cost function and its gradient. However, the cost function should be represented by binary variables (i.e., 0 or 1) for quantum annealers. In this study, we represent a real number with

Experiments conducted in this study. Outer loop indicates that the trajectory is updated during 4DVAR. Inner loop indicates that the analysis increments are computed iteratively by the quasi-Newton method with the BFGS formula.

This study performs twin idealized experiments using the 40-variable Lorenz 96 model, which has been used widely in theoretical data assimilation studies (e.g., Anderson, 2011; Whitaker and Hamill, 2002; Miyoshi, 2011; Kotsuki et al., 2017). The Lorenz 96 model is defined as follows:

As the experimental environment of the Phy-QA, we use D-Wave Advantage System 4.1 which consists of 5627 physical qubits and 177 logical qubits, respectively (D-Wave, 2022). Because the D-Wave Advantage performs computations using the logical qubits, the 177 logical qubits are available for computations. Because of this limitation, we represent a real number (an analysis increment for one variable of the Lorenz 96 model) with four logical qubits (

For the simulated quantum annealer, we use the Fixstars Amplify simulated quantum annealer (Fixstars Amplify Annealing Engine), which incorporates

Figure 2a and b show a comparison of the mean analysis and 2 d forecast root mean square errors (RMSEs) with respect to the nature run for the four different approaches. As anticipated, the NL-BFGS achieved the lower RMSEs as it did not approximate the original cost function. Approximating the nonlinear operations of the original cost function led to a slight increase in the analysis and forecast RMSEs, as observed in L-BFGS. Solving QUBO using the quantum annealers resulted in reduced analysis errors compared to the first guess, as demonstrated by the Sim-QA and Phy-QA. Here, tunable parameters for quantum annealers, namely the scaling factor,

On the other hand, Phy-QA demonstrated the fastest execution time among the four approaches (Fig. 2c). Here, NL-BFGS exhibited the longest computation time due to the iterative updates of the trajectory and its tangent linear and adjoint models. In contrast, L-BFGS, which retains the first-guess-based trajectory, was significantly faster than NL-BFGS. Sim-QA required a longer execution time than Phy-QA, presumably because GPU-based simulated quantum annealers involve computations for the artificial emulation of quantum effects. The D-Wave Phy-QA obtained a significant reduction in computation time compared to the other three approaches (NL-BFGS, L-BFGS, and Sim-QA), taking less than 0.05 s to find a solution.

It should be noted that there are slight differences in analysis and forecast RMSEs between NL-BFGS and L-BFGS in Fig. 2a and b. The degradations of L-BFGS with respect to NL-BFGS would be more pronounced for stronger nonlinear cases, such as those with longer time windows of 4DVAR, since Eqs. (9)–(13) directly simplify the nonlinear operator in 4DVAR to a linear operator. Here, our quantum data assimilation solves the QUBO problem (Eq. 21), which is an approximation of the cost function solved in L-BFGS (Eq. 14). Therefore, Sim-QA and Phy-QA would worsen too for stronger nonlinear cases.

An illustration of data assimilations from an arbitrarily selected first guess. The black star and the green circle indicate the truth and the first guess, respectively. Blue, red, orange, and magenta triangles are the analyses of NL-BFGS, L-BFGS, Sim-QA, and Phy-QA, respectively. Blue and red lines represent the analysis updates over iterations for NL-BFGS and L-BFGS, whose internal analyses are indicated by the cross marks. Red and blue contours show the cost functions with and without the linearization (L-QUO and NL-QUO), respectively. These cost functions were computed in a grid search for 2 variables (

Figure 3 provides an arbitrarily selected data assimilation example, where the cost functions of NL-QUO and L-QUO are depicted by blue and red contour lines, respectively. Note that Fig. 3 shows an example of analysis while Fig. 2a provides the mean RMSEs averaged over 50 data assimilations. The BFGS-based 4DVAR (NL-BFGS and L-BFGS) gradually converged towards their respective analyses through iterative updates. Consequently, both NL-BFGS and L-BFGS yielded analyses that are closer to the minima of their respective cost functions compared to the first guess (blue and red triangles in Fig. 3). In contrast, Sim-QA and Phy-QA produced a single analysis each as they do not involve iterations. In this specific example, Sim-QA, aiming to minimize L-QUBO, generated an analysis (as a yellow triangle) that was distant from the bottom of L-QUO. Conversely, the analysis produced by Phy-QA (a magenta triangle) is closer to the minima of NL-QUO. Notably, despite solving the same L-QUBO problem, Sim-QA and Phy-QA yielded greatly different analyses in this example. This discrepancy is presumed to be a result of stochastic quantum effects, which will be further investigated in the next subsection.

Sensitivity of

It is important to mention that the D-Wave physical quantum annealer produces non-deterministic outputs due to stochastic quantum effects. Therefore, the quantum annealer includes a tunable parameter called num_reads, which defines the number of states (output solutions) to be read from the solver. Generally, a larger value of num_reads increases the probability of obtaining a better solution. Here, the better solution, which results in smaller Hamiltonian

Sensitivity of analysis RMSEs to the scaling factor

The scaling factor

This study proposed the quantum data assimilation which solves data assimilation problems on quantum annealing machines. The main results of this investigation are as follows:

We reformulated the data assimilation problem into the quadratic unconstrained binary optimization (QUBO) problem so that quantum annealing machines can solve data assimilation.

Using the 40-variable Lorenz model, we succeeded in solving data assimilation on quantum annealers for the first time. The results of our experiments were highly promising, demonstrating that the quantum annealers can yield an analysis whose accuracy is comparable to the conventional quasi-Newton-based iterative approach.

The D-Wave physical quantum annealing machine needed an execution time of less than 0.05 s, which is significantly smaller than conventional approaches.

Since the D-Wave physical quantum annealer produces non-deterministic outputs due to stochastic quantum effects, reading out solutions multiple times was beneficial in achieving stable and improved analyses.

The scaling factor for quantum data assimilation is an important parameter for regulating analysis accuracy in our configuration. Due to the stochastic quantum effects, the optimal scaling factor for D-Wave's physical quantum annealing machine was different from the simulated quantum annealer.

We anticipate that our findings will inspire future developments in the application of quantum technologies to advance data assimilation to reach a deeper understanding and improved predictions of real-world complex systems in NWP and beyond. In addition, our work would advance the practical applications of quantum annealing machines in solving complex optimization problems in Earth science.

A conceptual image of the quantum annealing, showing

This Appendix describes how quantum annealers reach the solutions of the QUBO problems. First of all, it should be noted that there are two kinds of quantum computers at this moment: quantum annealing machines and quantum gate machines. Quantum gate machines are general-purpose quantum devices capable of performing a wide range of quantum computations. The quantum gate machines enable users to design quantum circuits which manipulate qubits through gate operations to solve complex problems efficiently.

Quantum annealers are devices specialized for solving optimization problems written by the QUBO problem or Ising model. Here, a problem written by the Ising model can be reformulated to a mathematically equivalent QUBO problem and vice versa. Figure A1 provides a conceptual image of quantum annealing. Users can submit jobs (i.e., problems written by the QUBO or Ising model) to quantum annealers through an application programming interface (API) (Fig. A1a). This study used the Fixstars Amplify software development kit (known as Amplify SDK) as an API to submit jobs to D-Wave's physical quantum annealer and the Fixstars Amplify simulated quantum annealer. Figure A1b shows the functions of the control device, quantum processing unit (QPU), and measuring device, respectively. The control device regulates magnetic fields to tune the Hamiltonian of the quantum system, guiding to a low-energy state which corresponds to an optimal solution of QUBO. In QPU, quantum effects (superposition and entanglement) are leveraged to explore potential solutions of the QUBO problem through quantum annealing (Fig. A1c). The measuring device reads the final state of the QPU. Here, num_reads defines the number of states to be read from the QPU by the measuring device. Finally, a user can obtain a solution (binary vector

The code that supports the findings of this study is available from the corresponding author upon reasonable request.

All of the data and codes used in this study are stored for 5 years at Chiba University and are available from the corresponding author upon request.

SK developed the methodology of the study, FK employed Lorenz 96 experiments, and MO conducted experiments on quantum annealers.

The contact author has declared that none of the authors has any competing interests.

Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this paper. While Copernicus Publications makes every effort to include appropriate place names, the final responsibility lies with the authors.

The authors thank project members of the Japan Science and Technology Agency Moonshot R&D for fruitful discussions.

This study was partly supported by the Japan Science and Technology Agency Moonshot R&D (grant nos. JPMJMS2284 and JPMJMS2389) and PRESTO (grant no. JPMJPR1924) programs, the Japan Society for the Promotion of Science (JSPS) via KAKENHI (grant nos. JP21H04571, JP21H05002, and JP22K18821), and the IAAR Research Support Program of Chiba University.

This paper was edited by Amit Apte and reviewed by two anonymous referees.