We provide here a brief review of the ARFIMA model. More details are given in Appendix .

An ARFIMA model is given by Φ(B)(1-B)dXt=Θ(B)εt. We define the backshift operator B, where BXt = Xt-1, and powers of B are defined iteratively: BkXt = Bk-1Xt-1 = ⋯ = Xt-k. Φ is the autoregressive component and Θ is the moving average component and constitute the short-memory components of the ARFIMA model. These are defined in more detail in Appendix  and in .

For now, we restrict our attention to a Bayesian analysis of an ARFIMA(0,d,0) process, having no short-ranged ARMA components (p = q = 0), placing emphasis squarely on the memory parameter d. As we explain in our Appendix, the resulting process is identical to a fractionally integrated processes with memory parameter d.

Here we develop an efficient and new scheme for evaluating the (log) likelihood, via approximation. Throughout, the reader should suppose that we have observed the vector x = (x1, …, xn)⊤ as a realization of a stationary, causal and invertible ARFIMA(0,d,0) process {Xt} with mean μ ∈ R. The innovations will be assumed to be independent, and taken from a zero-mean location-scale probability density f(⋅; 0, σ, λ), which means the density can be written as f(x; δ, σ, λ) ≡ 1σf(x-δσ; 0, 1, λ). The parameters δ and σ are called the location and scale parameters, respectively. The m-dimensional λ is a shape parameter (if it exists, i.e., m > 0). A common example is the Gaussian N(μ, σ2), where δ ≡ μ and there is no λ. We classify the four parameters μ, σ, λ, and d into three distinct classes: (1) the mean of process, μ; (2) innovation distribution parameters, υ = (σ, λ); and (3) memory structure, d. Together, ψ = (μ, υ, ω), where ω will later encompass the short-range parameters p and q.

Our proposed likelihood approximation uses a truncated autoregressive model (AR) (∞) approximation (cf. ). We first re-write the AR(∞) approximation of ARFIMA(0,d,0) to incorporate the unknown parameter μ, and drop the (d) superscript for convenience: Xt - μ = εt - ∑k=1∞πk(Xt-k - μ). Then we truncate this AR(∞) representation to obtain an AR(P) one, with P large enough to retain low frequency effects, e.g., P = n. We denote ΠP = ∑k=0Pπk and, with π0 = 1, rearrange terms to obtain the following modified model: Xt=εt+ΠPμ-∑k=1PπkXt-k.

It is now possible to write down a conditional likelihood. For convenience the notation xk = (x1, …, xk)⊤ for k = 1, …, n will be used (and x0 is interpreted as appropriate where necessary). Denote the unobserved P vector of random variables (x1-P, …, x-1, x0)⊤ by xA (in the Bayesian context these will be auxiliary, hence “A”). Consider the likelihood L(x|ψ) as a joint density, which can be factorized as a product of conditionals. Writing gt(xt|xt-1, ψ) for the density of Xt conditional on xt-1, we obtain L(x|ψ) = ∏t=1ngt(xt|xt-1, ψ).

This is still of little use because the gt may have a complicated form. However, by further conditioning on xA, and writing ht(xt|xA, xt-1, ψ) for the density of Xt conditional on xt-1 and xA, we obtain L(x|ψ, xA) = ∏t=1nht(xt|xA, xt-1, ψ). Returning to Eq. () observe that, conditional on both the observed and unobserved past values, Xt is simply distributed according to the innovations' density f with a suitable change in location: Xt|xt-1, xA ∼ f(⋅; [ΠPμ - ∑k=1Pπkxt-k], σ, λ). Then using location-scale representation: htxt|xA,xt-1,ψ≈fxt;ΠPμ-∑k=1Pπkxt-k,σ,λ≡1σfct-ΠPμσ;0,1,λ,wherect=∑k=0Pπkxt-k,t=1,…,n. Therefore, L(x|ψ, xA) ≈ σ-n∏t=1nfct-ΠPμσ;λ, or equivalently ℓx|ψ,xA≈-nlog⁡σ+∑t=1nlog⁡fct-ΠPμσ;λ.

Evaluating this expression efficiently depends upon efficient calculation of c = (c1, …, cn)t and log⁡f(⋅). From Eq. (), c is a convolution of the augmented data, (x, xA), and coefficients depending on d, which can be evaluated quickly in the R language for statistical computing via convolve via fast Fourier transform (FFT). Consequently, evaluation of the conditional on xA likelihood in the Gaussian case costs only O(nlog⁡n) – a clear improvement over the exact method. Obtaining the unconditional likelihood requires marginalization over xA, which is analytically infeasible. However, this conditional form will suffice in the context of our Bayesian inferential scheme, presented below.

Recall that short-memory components of an ARFIMA process are defined by the AR and moving average (MA) polynomials, Φ and Θ, respectively (see Sect. ). Here, we distinguish between the polynomial, Φ, and the vector of its coefficients, ϕ = (ϕ1, …, ϕp). When the polynomial degree is required explicitly, bracketed superscripts will be used: Φ(p), ϕ(p), Θ(p), θ(p).

We combine the short-memory parameters ϕ and θ with d to create a single memory parameter, ω = (ϕ, θ, d). For a given unit-variance ARFIMA(p,d,q) process, we denote its autocovariance (ACV) by γω(⋅), with γd(⋅) and γϕ, θ(⋅) those of the relevant unit-variance ARFIMA(0,d,0) and ARMA(p,q) processes, respectively. The spectral density function (SDF) of the unit-variance ARFIMA(p,d,q) process is written as fω(⋅), and its covariance matrix is Σω.

An exact likelihood evaluation requires an explicit calculation of the ACV γω(⋅); however, there is no simple closed form for arbitrary ARFIMA processes. Fortunately, our proposed approximate likelihood method of section can be ported over directly. Given the coefficients {πk(d)} and polynomials Φ and Θ, it is straightforward to calculate the {πk(ω)} coefficients required by again applying the numerical methods of Sect. 3.3.

To focus the exposition, consider the simple, yet useful, ARFIMA(1,d,0) model where the full memory parameter is ω = (d, ϕ1). Because the parameter spaces of d and ϕ1 are independent, it is simplest to update each of these parameters separately; d with the methods of Sect.  and ϕ1 similarly: ξϕ1|ϕ1 ∼ N(-1,1)(ϕ1, σϕ12), for some σϕ12. In practice however, the posteriors of d and ϕ1 typically exhibit significant correlation so independent proposals are inefficient. One solution would be to parametrize to some d* and orthogonal ϕ2*, but the interpretation of d* would not be clear. An alternative to explicit reparametrization is to update the parameters jointly, but in such a way that proposals are aligned with the correlation structure. This will ensure a reasonable acceptance rate and mixing.

To propose parameters in the manner described above, a two-dimensional, suitably truncated Gaussian random walk, with covariance matrix aligned with the posterior covariance, is required. To make proposals of this sort, and indeed for arbitrary ω in larger p and q cases, requires sampling from a hypercuboid-truncated multivariate normal (MVN) Nr(a,b)(ω, Σω), where (a, b) describe the coordinates of the hypercube. We find that rejection sampling-based unconstrained similarly parametrized MVN samples (e.g., using mvtnorm, ) works well, because in the RW setup the mode of the distribution always lies inside the hypercuboid. Returning to the specific ARFIMA(1,d,0) case, r = 2, b = (0.5, 1), and a = -b are appropriate choices. Calculation of the MH acceptance ratio Aω(ω, ξω) is trivial; it simply requires numerical evaluation of Φr(N)(⋅; ⋅, Σω), e.g., via mvtnorm, since the ratios of hypercuboid normalization terms would cancel. We find that initial values ϕ[0] chosen uniformly in C1; i.e., the interval (-1, 1), and d[0] from {-0.4, -0.2, 0, 0.2, 0.4} work well. Any choice of prior for ω can be made, although we prefer flat (proper) priors.

The only technical difficulty is the choice of proposal covariance matrix Σω. Ideally, it would be aligned with the posterior covariance; however, this is not a priori known. We find that running a pilot chain with independent proposals via Nr(a,b)(ω, σω2Ir) can help choose a Σω. A rescaled version of the sample covariance matrix from the pilot posterior chain, following , works well (see Sect. ).

We now expand the parameter space to include models M ∈ M, the set of ARFIMA models with p and q short-memory parameters, indexing the size of the parameter space Ψ(M). For our trans-dimensional moves, we only consider adjacent models, on which we will be more specific later. For now, note that the choice of bijective function mapping between model spaces (whose Jacobian term appears in the acceptance ratio) is crucial to the success of the sampler. To illustrate, consider transforming from Φ(p+1) ∈ Cp+1 down to Φ(p) ∈ Cp. This turns out to be a non-trivial problem, however, because (for p > 1) Cp has a very complicated shape. The most natural map would be (ϕ1, …, ϕp, ϕp+1) ⟼ (ϕ1, …, ϕp). However, there is no guarantee that the image will lie in Cp. Even if the model dimension is fixed, difficulties are still encountered; a natural proposal method would be to update each component of ϕ separately but, because of the awkward shape of Cp, the allowable values for each component are a complicated function of the others. Non-trivial proposals are required.

A potential approach is to parametrize in terms of the inverse roots (poles) of Φ, as advocated by : by writing Φ(z) = ∏i=1p(1 - αiz), we have ϕ(p) ∈ Cp ⟺ |αi| < 1 for all i. This looks attractive because it transforms Cp into Dp = D × ⋯ × D (p times) where D is the open unit disc, which is easy to sample from. But this method has serious drawbacks when we consider the RJ step. To decrease dimension, the natural map would be to remove one of the roots from the polynomial. But because it is assumed that Φ has real coefficients (otherwise the model has no realistic interpretation), any complex αi must appear as conjugate pairs. There is then no obvious way to remove a root; a contrived method might be to remove the conjugate pair and replace it with a real root with the same modulus; however, it is unclear how this new polynomial is related to the original, and to other aspects of the process, like ACV.

Standard MCMC diagnostics were used throughout to ensure, and tune for, good mixing. Because d is the parameter of primary interest, the initial values d[0] will be chosen to systematically cover its parameter space, usually starting five chains at the regularly spaced points {-0.4, -0.2, 0, 0.2, 0.4}. Initial values for other parameters are not varied: μ will start at the sample mean x‾; σ at the sample standard deviation of the observed series x.

Known form: we first consider the MCMC algorithm from Sect.  for sampling under an ARFIMA(1,d,0) model where the full memory parameter is ω = (d, ϕ1). Recall that method involved proposals from a hypercuboid MVN using a pilot-tuned covariance matrix. Also recall that it is a special case of the reparametrized method from Sect. .

In general, this method works very well; two example outputs are presented in Fig. , under two similar data-generating mechanisms.

Figure a shows relatively mild correlation (ρ = 0.21) compared with Fig. b, which shows strong correlation (ρ = 0.91). This differential behavior can be explained heuristically by considering the differing data-generating values. For the process in Fig. a the short-memory and long-memory components exhibit their effects at opposite ends of the spectrum; see Fig. a. The resulting ARFIMA spectrum, with peaks at either end, makes it easy to distinguish between short and long-memory effects, and consequently the posteriors of d and ϕ are largely uncorrelated. In contrast, the parameters of the process in Fig. b express their behavior at the same end of the spectrum. With negative d these effects partially cancel each other out, except very near the origin where the negative memory effect dominates; see Fig. b. Distinguishing between the effects of ϕ and d is much more difficult in this case; consequently the posteriors are much more dependent.

In cases where there is significant correlation between d and ϕ, it arguably makes little sense to consider only the marginal posterior distribution of d. For example the 95 % credibility interval for d from Fig. b is (-0.473, -0.247), and the corresponding interval for ϕ is (-0.910, -0.753), yet these clearly give a rather pessimistic view of our joint knowledge about d and ϕ; see Fig. c. In theory an ellipsoidal credibility set could be constructed although this is clearly less practical when ∼ ω > 2.

Unknown form: the RJ scheme outlined in Sect.  works well for data simulated with p and q up to 3. The marginal posteriors for d are generally roughly centered around dI (the data-generating value) and the modal posterior model probability is usually the correct one. To illustrate, consider again the two example data-generating contexts used above.

For both series, kernel density for the marginal posterior for d are plotted in Fig. a and b, together with the equivalent density estimated assuming unknown model orders.

Notice how the densities obtained via the RJ method are very close to those obtained assuming p = 1 and q = 0. The former are slightly more heavy tailed, reflecting a greater level of uncertainty about d. Interestingly, the corresponding plots for the posteriors of μ and σ do not appear to exhibit this effect; see Fig. c and d. The posterior model probabilities are presented in Table , showing that the correct modes are being picked up consistently.

As a test of the robustness of the method, consider a complicated short-memory input combined with a heavy-tailed α-stable innovations distribution. Specifically, the time series that will be used is the following ARFIMA(2,d,1) process 1-916B2(1-B)0.25Xt=1+13Bεt,whereεt∼S1.75,0. For more details, see Sect. 7.1. The marginal posterior densities of d and α are presented in Fig. .

Performance looks good despite the complicated structure. The posterior estimate for d is d^(B) = 0.22, with 95 % CI (0.04, 0.41). Although this interval is admittedly rather wide, it is reasonably clear that long memory is present in the signal. The corresponding interval for α is (1.71, 1.88) with estimate α^(B) = 1.79. Finally, we see from Table  that the algorithm is very rarely in the wrong model.

Because of the fundamental importance of the Nile river to the civilizations it has supported, local rulers kept measurements of the annual maximal and minimal heights obtained by the river at certain points (called gauges). The longest uninterrupted sequence of recordings is from the Roda gauge (near Cairo), between AD 622 and 1284 (n = 663).

There is evidence e.g., that the sequence is not actually homogeneous.

There is increasing evidence that surface air temperatures posses long memory but long time series are needed to get robust results. The CET index is a famous measure of the monthly mean temperature in an area of southern-central England dating back to 1659 . Given to a precision of 0.5 ∘C prior to 1699 and 0.1 ∘C thereafter, the index is considered to be the longest reliable known temperature record from station data. As expected, the CET exhibits a significant seasonal signal, at least some of which must be considered as deterministic. Following the approach of , the index is first deseasonalized using the additive “STL” method . This deseasonalized CET index is shown in Fig. .

The estimated seasonal function S(t) that was removed is shown in Fig. a. The spectrum of the deseasonalized process is shown in Fig. b. D denotes the seasonal long-memory parameter. Notice that, in addition to the obvious spectral peak at the origin, there still remains a noticeable peak at the monthly frequency ω = π6. However, there are no further peaks in the spectrum, which would appear to rule out a seasonal ARFIMA (SARFIMA) model. These observations therefore suggest that a simple two-frequency Gegenbauer(d,D;π6)2 process might be an appropriate model. See Appendix  for more details about seasonal long memory.