Image-model Coupling: a Simple Information Theoretic Perspective for Image Sequences

Images are widely used to visualise physical processes. Models may be developed which attempt to repli-cate those processes and their effects. The technique of coupling model output to images, which is here called " image-model coupling " , may be used to help understand the underlying physical processes, and better understand the limitations of the models. An information theoretic framework is presented for image-model coupling in the context of communication along a discrete channel. The physical process may be regarded as a transmitter of images and the model as part of a receiver which decodes or recognises those images. Image-model coupling may therefore be interpreted as image recognition. Of interest are physical processes which exhibit " memory ". The response of such a system is not only dependent on the current values of driver variables, but also on the recent history of drivers and/or system description. Examples of such systems in geophysics include the ionosphere and Earth's climate. The discrete channel model is used to help derive expressions for matching images and model output , and help analyse the coupling.


Introduction
Images describe what is present in a scene, and mathematical or empirical models may be developed which attempt to replicate the underlying processes which produce those images.Given an image, it is interesting to find the model input producing output which best matches the image.In this way useful knowledge concerning the mechanisms producing that image may be discerned.Such image-model coupling, in the context of sequences of images and systems with memory, is described below.The particular example application chosen is the coupling of images of the ionosphere and the output of ionospheric models.This is a challenging problem because the ionosphere (Hargreaves, 2003) is a nonlinear system with regard to the response of its electron content to its driver variables.Furthermore, the system has "memory" since its response at a given time depends not only on the current drivers but also on the recent history.
There are various questions to consider.What objective function should be used to describe the match between an image sequence and model output?More generally, how can competing models be compared in a quantitative manner?And how can the importance of driver variables, for replicating image sequences, be assessed?
To attempt to answer these questions in a consistent manner, image-model coupling is presented within a simple information theoretic context as transmission along a discrete channel.The simpler and more familiar discrete memoryless channel is a special case suitable for systems which do not exhibit memory.This approach clearly separates the true underlying real-world process producing the image sequence and the proposed model, and gives a framework to help identify assumptions in the proposed model and possible sources Published by Copernicus Publications on behalf of the European Geosciences Union and the American Geophysical Union.N. D. Smith et al.: Image-model coupling, a simple information theoretic perspective of error in coupling model output to images.The objective function used for coupling is statistical in nature; with an increased availability of data, it should be possible to derive objective functions which improve the coupling.The context encourages the use of "tools" drawn from information theory and statistical modelling.
The approach borrows heavily from joint source-channel decoding (JSCD), e.g.Garcia-Frias and Villasenor (2001); Ferrari et al. (2005); Link and Kallel (2000) and speech recognition, e.g.Rabiner and Juang (1993), where both maximum a-posteriori (MAP) decoding and evaluation in terms of error rate are commonplace.Modelling channel noise is important for communication systems in general, e.g.Burlina and Alajaji (1998); Beaulieu (1991).Our approach is very similar to the analysis of stochastic channels in Ferrari et al. (2005), except we have here assumed specific conditional independences.This paper seeks to apply concepts from information theory and speech recognition to the modelling of geophysical systems; memory is often important for such systems.The ionosphere (e.g.see Hargreaves, 2003) is used as an illustration in this text.Memory is also important for Earth's climate, e.g.Maraun et al. (2004).The statistical aspects of our framework are embedded in graphical models and the generic Bayesian approach; regarding geophysical systems, note that graphical models, hierarchical Bayesian models and hidden Markov models (HMMs) have previously been applied, e.g.Ihler et al. (2007); Wikle et al. (1998);Beyreuther and Wassermann (2008).There is significant overlap with data assimilation, e.g.Wikle and Berliner (2007); however data assimilation is primarily interested in using models to improve the prediction of observations rather than to decode the values of driver variables.Ignoring this distinction, image-model coupling is essentially a form of data assimilation.Our distinct separation of real-world process and proposed model is similar to that in Duane et al. (2006), where data assimilation is viewed in terms of the synchronisation of two dynamical systems via a noisy communication channel.There are similarities between the framework for data assimilation applied to the ionosphere in Guo et al. (2003) and our approach, but in Guo et al. (2003) there is no development in terms of a discrete communication channel.As far as the authors are aware, the main contribution of this paper is in the application of concepts from JSCD, speech recognition and statistical modelling to the "decoding" of the values of driver variables for geophysical systems, in partic-ular by highlighting the role of memory and missing variables, and in analysing the coupling.While the concepts are not new, the authors hope the approach may encourage extra insight in modelling geophysical systems.
The paper is organised as follows.First, Sect. 2 introduces discrete channel models with and without memory; the true underlying process producing the image sequence is considered as a transmitter, and the proposed model is part of a receiver which interprets, or decodes, the image sequence.Section 3 then describes the receiver in more detail; the codebook, noise and state transition models, objective function for matching, and the search mechanism.In particular, the assumptions implicit in applying simple sum square error minimisation are detailed.Sections 4 and 5 respectively introduce expressions to compare alternative models, and estimate the sensitivity of image sequences to different driver variables.Finally some discussion and conclusions follow in Sects.6 and 7.

Communication channel framework
Image-model coupling may be viewed from an information theoretic perspective as communication via a discrete channel (e.g.see MacKay, 2004;Abramson, 1963;Khinchin, 1957).The true-world process generating the images is viewed as a transmitter, and the model which is used to interpret the images is part of the receiver.Some form of synchronous "online" decoding is attractive for continuous communication, i.e. continuous "online" image-model coupling.
In the context of ionospheric modelling, the true ionosphere acts as a transmitter and encodes physical driver variables as an image of the ionosphere, where the image is simply an often incomplete description of the ionosphere.The image may be in the form of electron densities or their line integrals.The ionospheric model is part of a decoder which attempts to recover the values of those driver variables.This channel approach allows us to derive an objective function for matching a temporal sequence of images with model output.The following analysis concerns systems such as the ionosphere which are not memoryless, so receivers which assume simple discrete memoryless channel models may not be accurate.However, for reasons of tractability, such receivers may be applied, though it is useful to understand the limitations and assumptions in so doing.In the following, "TX" denotes the transmitter and "RXq" the receiver for q∈{3, 2, 1}.
n RXq (t) x RXq (t) δt 2δt hcδt hcδt Fig. 1.Image-model coupling as a discrete channel model, where the true real-world process is an encoder, δt is the duration of a timestep when used to describe delay in a temporal buffer, and RXq, q∈{3, 2, 1} denotes different receivers.

Transmitter (TX)
Figure 1 describes the true real-world process assumed to generate the images.The process is assumed driven.The driver variables of interest are recorded at time t as u TX (t)∈U TX , where U TX is a discrete, typically open set1 .For the ionosphere, driver variables of interest may include measurements of solar or geomagnetic activity.In addition, there are latent driver variables u TX (t)∈U TX which are not measured and are typically unknown or not of direct interest to the modeller.Again, U TX is a discrete set.The current system is fully described by z TX (t)∈Z TX and z TX (t)∈Z TX , which respectively describe those variables which form the image, and the complementary set of variables required to complete the full description.For convenience, define discrete sets Z TX ⊆Z and Z TX ⊆Z .The real-world process is assumed viewed as a codebook which implements the deterministic mapping, for some fixed and known channel memory length2 h c ∈N 0 (for channel memory, e.g.see Khinchin, 1957), where t is the timestep index and h c is expressed in timesteps, and for example, Similar abbreviations are used elsewhere for time-ordered temporal sequences of vectors.Hence z TX (t) is fully determined by the present and past driver variables, both known and latent, and an initial complete description.Hence there are no variables beyond those in the domain of the mapping which cause variation in z TX (t).The assumption of the deterministic mapping is thought reasonable due to the inclusion of all driver variables and description variables in the domain.In the case of the ionosphere, the dependency on initialisation is assumed since the ionosphere is not memoryless; its description may evolve differently over time under the action of the same sequence of drivers depending on the initial distribution of plasma.
There is also memory in the source.Letting h s ∈N 0 denote the length of source memory in timesteps, then it is convenient to define the state, where h= max[h s −1, h c ], where x TX (t)∈X TX , and where This definition of state allows the transmitter to be modelled as a hidden Markov process (see Sect.The actual image measured or recorded at time t is z(t)∈Z.This image is related to the partial description z TX (t) by, where n TX (t)∈Z is additive noise describing error in the measuring devices.Unfortunately, if an image is incomplete, it is sometimes necessary to complete the image, for example using tomographic reconstruction.For the purposes of this analysis, such images are regarded as if directly imaged by a device, and the error in the reconstruction included into the noise process n TX (t).Both the true real-world process and noise source may be nonstationary; however for the estimation of the statistical models described later, properties of stationarity and ergodicity (Korn and Korn, 1968) are convenient.The noise signal n TX (t) is not transmitted independently along the channel, only the image z(t).In practice, it is usual to consider a finite length sequence of images, for example T images z(1, T ).

Level 3 receiver (RX3)
With infinite knowledge, it is possible to construct a receiver which implements the reverse process to the transmitter.Hence the receiver first "denoises" the image, where n RX3 (t)∈Z, z RX3 (t)∈Z RX3 , and Z RX3 is typically a discrete set denoting the range of the receiver codebook.The codebook implements deterministic mappings of form, but is used in the reverse direction.The codebook is typically implemented by a deterministic mathematical or empirical model.The noise source n RX3 (t) should model the measurement noise in the imaging devices.This receiver is unrealisable, but is included since it permits decoding with the lowest possible error rate.Decoding is described more fully in Sect.3.4.For clarity, the receiver is here called a level 3 receiver, where the higher the level, the deeper the conditional dependencies in the receiver codebook.For convenience, a receiver state is defined, where

Level 2 receiver (RX2)
This is similar to the level 3 receiver with differences in the codebook mappings and notation.Denoising is, where n RX2 (t)∈Z and z RX2 (t)∈Z RX2 .The codebook implements deterministic mappings of form, where the unmeasured or unknown driver and description variables in U TX and Z TX have been omitted.The stochastic variation in z(t) due to these variables is instead incorporated into a more complicated noise source n RX2 (t).The noise source no longer models the error in measurement devices alone, but also the stochastic variation due to the omitted variables.Again, define a state, where

Level 1 receiver (RX1)
This is similar to the level 3 and 2 receivers with differences in the codebook mappings and notation.Denoising is, where n RX1 (t)∈Z and z RX1 (t)∈Z RX1 .The codebook implements deterministic mappings of form, Each codebook entry z RX1 (t) may be regarded as "typical" for its driver variables, in a similar manner to which the mean of a Gaussian distribution is typical of samples drawn from that Gaussian.The noise source n RX1 (t) should now also describe the stochastic variation due to different initialisations and histories of driver variables.The present drivers form the state so x RX1 (t)=u RX1 (t) where x RX1 (t)∈X RX1 =U RX1 .

Perfect image-model coupling
In this analysis, perfect image-model coupling is the transmission of a sequence of driver variables, without loss, via the true real-world process as encoder, the images as the transmitted message, and the proposed model as part of the decoder.Ideally u TX (t)=u RXq (t).For the example of the level 3 receiver, assume that, -the codebook mappings are identical, i.e. z −1 RX3 (t) • z TX (t)=I , where I is the identity map, -the domain of the receiver and transmitter codebooks are identically descriptive, i.e.U RX3 =U TX , -the transmitter noise source is correctly modelled by the receiver noise source, i.e.P RX3 (n RX3 (t))=P TX (n TX (t)), ∀t, and, -processes and models implicit in the transmitter and receiver are assumed stationary.
The first two conditions assume the proposed codebook is correct, the third that the noise model is correct.Unfortunately, even under these conditions where the distribution of noise n RX3 (t) is correct, the particular sample drawn from that noise distribution remains unknown.For this reason, even in the case of the level 3 receiver, perfect coupling may not be achievable.In many cases, the driver variables u TX (t) can only be recovered in the sense of maximum a-posteriori (MAP) or other estimates.For either of the level 3, 2 or 1 receivers, perfect coupling would only be possible if the transmitter and receiver were identical, and either noise samples were transmitted independently between the transmitter and receiver along a separate noiseless channel, or the supports of the noise distributions were always strictly less than the distances between neighbouring entries in the ranges of the transmitter and receiver codebooks.Decoding is described more fully in Sect.3.

Receiver
The purpose of the receiver is to decode the image sequence z(1, T ) as a sequence of driver variables.Assume the receiver at level q implements this by first decoding the most appropriate state sequence through minimising a scalar objective function, subject to the constraint that there exists an underlying consistent driver sequence ûRXq (1, T )(z(1, T )), i.e. an extraction mapping should exist, This section explains how the receiver achieves this.The definition of the receiver requires the specification of (1) a codebook, both the mapping as implemented by a deterministic model and its domain, (2) a noise model, and (3) a state transition model.The decoder attempts to find the best match between each image and a member of the codebook.This requires careful selection of (4) the objective function to measure the "goodness of match", and (5) a search mechanism, often heuristic, to navigate the codebook and find the member with maximum "goodness of match".These components are described in the remainder of this section.

Codebook
For the level q receiver, the codebook may be viewed as the set of deterministic mappings, These may be implemented using lookup tables, but more typically by empirical or mathematical models.The codebook domain X RXq may also be constrained by the choice of model.For example an empirical model may impose lower and upper bounds on its driver variables, which in turn constrain the codebook domain.The domain may be quantised coarsely, or finely at machine precision.In ionospheric modelling, some codebooks may be more suitable for different tasks than others, e.g.some deterministic models may be better at describing high latitude processes while others may be more suitable for low latitude processes. www In effect {C RXq , N RXq } defines a stochastic version of the deterministic codebook.Indeed if the model involved in imagemodel coupling is stochastic, then there is no need to define C RXq and N RXq separately.Ideally the codebook and noise model should replicate the stochastic variation in the transmitter (i.e. the real-world process).However this is a challenging task since considerable complexity is expected in the real-world variation, as described in Appendix A.

State transition model
Another supplementary model must be supplied governing the transitions between consecutive states.This may again be fully specified by a set of PMFs.For a level q receiver, This state transition model must assign zero probability mass to those transitions which do not respect a temporally consistent sequence, e.g. for u RXq (1, T ).

Objective function
The objective function should be derived from a decision theoretic perspective.For each sample z(1, T ), the decision rule should seek to minimise the conditional risk (Duda et al., 2001), where x RXq (t)∈X RXq , ∀t∈[1, T ].The scalar function l(•, •) is the loss, and the conditional risk is expressed as the average loss over a posterior distribution in the receiver.An intuitive choice of loss function is the squared L2 norm of the difference between the two arguments.However, under such a "regression-based" loss, the conditional risk is expensive to compute if samples must be drawn from the posterior.A simpler "classification-based" loss may instead be used (Duda et al., 2001), This is preferable because it reduces computational cost since then, thereby avoiding the averaging operation.Hence an objective function may be expressed as follows, where logs are taken and unneccessary terms are discarded from the log posterior, The first of the two terms in the objective function models state-dependent noise, the second models memory in the state space (which includes memory in the driver source).Stochastic variation originates both in the channel and the source (e.g.see Khinchin, 1957).The first term is determined by the codebook C RXq and noise model N RXq , the second by the state transition model A RXq .The resultant decoder is the well-known maximum a-posteriori (MAP) decoder where the posterior acts as a measure of "goodness of match".The objective function may be compared with those derived in variational analysis in other applications such as meteorology (Daley, 1999).The remainder of this subsection considers how to obtain expressions for the objective function under different receivers, and details the assumptions implicit in decoding images using least sum square error optimisation.

Discrete channel with memory
All receivers RXq, q∈{3, 2, 1} may be modelled as hidden Markov processes of the form shown in Fig. 2, where the state is assumed to contain all information about the past which can influence the present and the future.
The noise process and state transition process are assumed stationary in time.The posterior is, where it is assumed that the state x RXq (0) is fully known, i.e. the history of driver variables and pre-noised images is known back to a timestep index of −h, if required.The objective function becomes, Both level 3 and level 2 receivers assume channel memory length and source memory length no greater than h and h+1 timesteps respectively.Of course, those memory lengths may in practice be reduced by constraining the PMFs in N RXq and A RXq .For example, channel memory length may be reduced to zero in favour of source memory alone.

Discrete memoryless channel
The discrete memoryless channel is well known and is worthy of further consideration.Assume the probability of a state is independent of the previous state.This is not possible for a level 3 or 2 receiver.For the level 1 receiver, The noise is still state-dependent though it is no longer Markov.This channel assumes there is no source memory, i.e. there is independence between successive values of driver variables.This independence assumption is severe and most probably unrealistic for real-world systems.Driver variables naturally change smoothly, albeit at a certain level of scale.For the ionosphere, on all but the shortest time scales, any measurement of incident solar radiation may appear discontinous during the arrival of a solar flare.However for the majority of time, this measurement varies smoothly between consecutive timesteps, and the state transition model should favour such smooth changes.Also the assumption that the image is conditionally independent of previous variables given the current driver variables is unrealistic for systems such as the ionosphere where plasma accumulates with time.The ionosphere's response, as illustrated by its description, under the action of a given set of drivers, will vary depending on its past driver values.However the discrete memoryless assumption is convenient and, at the risk of reduced decoding accuracy, the likelihood P RX1 (z(t)|u RX1 (t)) may be regarded as averaged over all possible histories.

Discrete memoryless channel with state-independent Gaussian noise
The objective function for the discrete memoryless channel, as given in Eq. ( 23), may be simplified to yield the well-known sum square error and sum weighted square error objective functions.It is useful to consider the additional assumptions.First, it is sometimes convenient to assume that the mapping u RX1 (t) →z RX1 (t) is injective ∀t∈[1, T ] (e.g.see "Mappings" by I. N. Sneddon in Sneddon, 1976).The injective assumption may not be unreasonable, particularly when a vector of few driver variables maps into a large image with many components.Then the objective function in Eq. ( 23) becomes, The prior P RX1 (u RX1 (t)) should reflect the frequency of occurrence of different values of driver variables over all possible histories.For example, for the ionosphere the prior may be calculated using measurements collected over a full solar cycle.
Next consider the noise process is state-independent and is stationary in time.Hence P RX1 (z(t)|z RX1 (t)) = δ(z(t) − z RX1 (t), n RX1 )P RX1 (n RX1 ), where δ(•, •) is the Kronecker delta, simplifying the noise model N RX1 considerably to the specification of a single PMF.This is unlikely for systems such as the ionosphere.Additionally, assume the noise model is a zero-mean discretised Gaussian so P RX1 (n RX1 ) = N (n RX1 ; 0, R) where R is the covariance matrix3 , and hence P RX1 (z(t)|z RX1 (t)) = N(z(t); z RX1 (t), R).Unfortunately, Gaussianity is unlikely for those description variables, for example line integrals of electron content in the case of the ionosphere, which are naturally nonnegative.
Finally, assume there is no prior preference in the driver variables so there is a uniform prior P RX1 (u RX1 (t)) over U RX1 .Again this is probably unreasonable for many real-world systems.For the case of the ionosphere, quiet space weather occurs much more frequently than stormy space weather, and the prior over driver variables should reflect this.The objective function for the discrete memoryless channel in Eq. ( 24) may now be simplified to the following, where irrelevant terms have been discarded, The more negative the objective function, the better is the "goodness of match".If the covariance matrix R is assumed diagonal, minimisation is equivalent to the conventional least sum weighted square error solution.The diagonal elements, i.e. weights, represent the relative "importance" of each component in the image.If each component is of equal importance, R may be set to Identity and the minimisation yields the least sum square error solution.

Search
Minimisation of the objective function over the full domain ⊗ T t=1 X RXq of the receiver codebook requires a search mechanism.The simplest approach is full grid search.However computational cost increases with the number of state variables, and coarser grid searches must often be introduced.Otherwise conventional derivative-based optimisation is recommended.However if the codebook mapping is implemented by some deterministic empirical model or when a mathematical model is very complicated, the derivative of an objective function of type f RXq (•) may not be known analytically.In this case, derivative-free optimisation techniques may then be required.Alternative approaches include numerical approximation of gradients (e.g.see the algorithms in Powell, 2007) and sampling-based methods.An example of a sampling scheme is simulated annealing (Salamon et al., 2002) which, although it converges to a global minimum, requires many evaluations of the objective function.A variant of simulated annealing is fast annealing (Salamon et al., 2002).Further comments on decoders are given in Sect.6.

Evaluation
It is sometimes useful to compare different codebooks for image-model coupling.Codebooks should not be compared in isolation, but in the context of the accompanying noise model, state transition model, objective function and search mechanism.
Given a single sample zl (1, T ) and a level 1 receiver, a possible estimate at sequence v RX1 (1, T ) is, noting x RX1 (1, T )=u RX1 (1, T ).In this case the Fisher information is estimated using the negative Hessian of the objective function.However such an estimate may be misleading since it is "tuned" to one particular image sequence only, i.e. all the conditional probability mass for observed sequences is assumed located at zl (1, T ).

Discussion
The selection of driver and image variables for U RXq and Z respectively is critical.If the selection is not sufficiently descriptive, then the noise distributions implied in the transmitter may be very broad due to the effect of latent variables.
Essentially, too much useful information may then be lost in the encoding process.Unfortunately, the choice of variables is often restricted by the particular receiver codebook, e.g.empirical or mathematical model, used.
In designing the receiver, the codebook is intuitively most important and effort should first be directed at improving this.The choice of noise model and state transition model should be data-dependent, since they should be learnt from data.This influences the choice of objective function and level of receiver.The simpler receivers should be more robust if there is a lack of good quality data.However it is important to understand the assumptions implicit in the simpler receivers, especially regarding the conditional independences.Of course, the level 3 and level 2 receivers reduce to the level 1 receiver if the level 1 assumptions hold in the transmitter.The choice of h s , h c and h may be driven by limitations in data rather than scientific knowledge.The inverse problem of decoding state variables xRXq (1, T ) for a noisy image sequence z(1, T ) has a unique solution providing the objective function has a single global minimum.
The receiver codebook has been defined using a single deterministic model.However multiple models may be used in parallel where the data fusion occurs at the level of z RXq (1, T ) or x RXq (1, T ).For ionospheric modelling, the fusion may additionally use geographic information where alternative ionospheric models are weighted differently at different global locations, for example according to their ability to model low latitude or polar/auroral processes.
The approach described above is limited.Our HMM enforces specific conditional dependencies; it is possible to apply different graphical models, e.g.Ihler et al. (2007).For example, the PMFs in our approach may be further constrained by incorporating spatial dependencies between variables, such as between pixels in an image, e.g.Burlina and Alajaji (1998); Link and Kallel (2000).When the channel is not memoryless, our approach also assumes additive Markov noise; other noise models are possible, for example semi-Markov noise (e.g.Chvosta and Reineker, 1999), or nonadditive noise.The approach is able to model intersymbol interference (ISI), as explained in Ferrari et al. (2005).It can also model the finite-state Markov channel (FSMC), e.g.Goldsmith and Varaiya (1996); Li and Collins (2007), where the driver variables form the system input.Implementing the decoder is challenging; a possible implementation is the Viterbi algorithm, e.g.Rabiner and Juang (1993); Kavčić and Moura (2000).However in practice, decoders may be too computationally expensive to implement unless the codebook mapping is computed apriori and stored in lookup tables, or the codebook mapping is very simple.Regarding decoders, it may be possible to incorporate concepts from JSCD such as interleavers, e.g.Li and Collins (2007), and the decoding algorithm, linked to Baum-Welch estimation, in Garcia-Frias and Villasenor (2001).Generally, it is difficult estimating PMFs with limited data, though concepts from speech recognition such as state and mixture tying and language model smoothing, e.g.Rabiner and Juang (1993), may prove beneficial.The accurate estimation of PMFs in the noise and state transition models is the main drawback of this approach.In practice, many simplifications may be required, in which case the receiver may degrade to a much simpler form.As detailed in Maraun et al. (2004), memory is often associated with the correlation function, and may be finite or infinite.It would be useful to analyse our approach by relating it to such a definition of memory.
Similar "receivers" have been developed in nonlinear and linear time series analysis.Examples from nonlinear time series analysis include the non-linear autoreggresive (NLAR) model (Chatfield, 2004), non-linear moving average (NLMA) model (Tong, 1990) and state-dependent model (SDM) (Priestley, 1988); however these examples assume there is no hidden state, equivalent to there being no channel noise.Also, Variable order Markov Models (VMMs) (Begleiter et al., 2004) model memory, but again there are no hidden states.The ARMA-filtered hidden Markov model (Michalek et al., 2000) may possibly be regarded as a special case of the level 2 receiver, at least in concept.
Besides the application to the ionosphere, the approach assuming memory may be useful for other geophysical systems which exhibit "sluggish responses", e.g.long-term climate modelling.As stated, a challenge for such systems is the selection of driver and image variables.For example, drivers for the ionosphere should include measures of solar and geomagnetic activity since it is known the sun and geomagnetic state of the Earth influence the distribution and movement of ionospheric plasma (Hargreaves, 2003).However if a 3dimensional map of electron density measurements is used www.nonlin-processes-geophys.net/16/197/2009/ Nonlin.Processes Geophys., 16,[197][198][199][200][201][202][203][204][205][206][207][208][209][210]2009 . Smith et al.: Image-model coupling, a simple information theoretic perspective Noise modelSince the codebook is deterministic, stochastic variability must be introduced via a supplementary noise model.For the level q receiver, the noise process may be fully specified via a set of probability mass functions (PMFs), N RXq = {P RXq (n RXq (t)|x RXq (t)), ∀n RXq (t)∈Z, ∀x RXq (t)∈X RXq }.