Approximation by Finite Mixtures of Continuous Density Functions That Vanish at Infinity

TrungTin Nguyen; Hien D. Nguyen; Faicel Chamroukhi; Geoffrey J. McLachlan

Abstract

Given sufficiently many components, it is often cited that finite mixture models can approximate any other probability density function (pdf) to an arbitrary degree of accuracy. Unfortunately, the nature of this approximation result is often left unclear. We prove that finite mixture models constructed from pdfs in $𝒞_{0}$ can be used to conduct approximation of various classes of approximands in a number of different modes. That is, we prove approximands in $𝒞_{0}$ can be uniformly approximated, approximands in $𝒞_{b}$ can be uniformly approximated on compact sets, and approximands in $ℒ_{p}$ can be approximated with respect to the $ℒ_{p}$ , for $p \in [1, \infty)$ . Furthermore, we also prove that measurable functions can be approximated, almost everywhere.

Keywords— Approximation theory, probability density functions, finite mixture models, Riemann summation, uniform approximation.

1 Introduction

Let $x$ be an element in the Euclidean space, defined by $ℝ^{n}$ and the norm $∥ \cdot ∥_{2}$ , for some $n \in ℕ$ . Let $f : ℝ^{n} \to ℝ$ be a function, such that $f \geq 0$ , everywhere, and $\int f d λ = 1$ , where $λ$ is the Lesbegue measure. We say that $f$ is a probability density function (pdf) on the domain $ℝ^{n}$ (an expression that we will drop, from hereon in). Let $g : ℝ^{n} \to ℝ$ be another pdf, and for each $m \in ℕ$ , define the functional class:

ℳ_{m}^{g}

=

{h : h ​ (x) = \sum_{i = 1}^{m} c_{i} ​ \frac{1}{σ_{i}^{n}} ​ g ​ (\frac{x - μ_{i}}{σ_{i}}), μ_{i} \in ℝ^{n} ​, ​ σ_{i} \in ℝ_{+} ​, ​ c \in 𝕊^{m - 1} ​, ​ i \in [m]} ​,

where $c^{⊤} = (c_{1}, \dots, c_{m})$ , $ℝ_{+} = (0, \infty)$ ,

𝕊^{m - 1} = {c \in ℝ^{m} : \sum_{i = 1}^{m} c_{i} = 1 ​ and ​ c_{i} \geq 0, \forall i \in [m]} ​,

$[m] = {1, \dots, m}$ , and ${(\cdot)}^{⊤}$ is the matrix transposition operator. We say that any $h \in ℳ_{m}^{g}$ is a $m -component$ location-scale finite mixture of the pdf $g$ .

The study of pdfs in the class $ℳ_{m}^{g}$ is an evergreen area of applied and technical research, in statistics. We point the interested reader to the many comprehensive books on the topic, such as [10],[35], [22], [20], [23], [14], [33], [24], and [15].

Much of the popularity of finite mixture models stem from the folk theorem, which states that for any density $f$ , there exists an $h \in ℳ_{m}^{g}$ , for some sufficiently large number of components $m \in ℕ$ , such that $h$ approximates $f$ arbitrarily closely, in some sense. Examples of this folk theorem come in statements such as: “provided the number of component densities is not bounded above, certain forms of mixture can be used to provide arbitrarily close approximation to a given probability distribution” [35, p. 50], “the [mixture] model forms can fit any distribution and significantly increase model fit” [37, p. 173], and “a mixture model can approximate almost any distribution” [39, p. 500]. Other statements conveying the same sentiment are reported in [28]. There is a sense of vagary in the reported statements, and little is ever made clear regarding the technical nature of the folk theorem.

In order to proceed, we require the following definitions. We say that $f$ is compactly supported on $𝕂 \subset ℝ^{n}$ , if $𝕂$ is compact and if $𝟏_{𝕂^{∁}} f = 0$ , where $𝟏_{𝕏}$ is the indicator function that takes value 1 when $x \in 𝕏$ and $0$ , elsewhere, and ${(\cdot)}^{∁}$ is the set complement operator (i.e., $𝕏^{∁} = ℝ^{n} \ 𝕏$ ). Here, $𝕏$ is a generic subset of $ℝ^{n}$ . Furthermore, we say that $f \in ℒ_{p} (𝕏)$ for any $1 \leq p < \infty$ , if

{‖ f ‖}_{ℒ_{p} ​ (𝕏)} = {(\int {| 𝟏_{𝕏} ​ f |}^{p} ​ d ​ λ)}^{1 / p} < \infty ​,

and for $p = \infty$ , if

{‖ f ‖}_{ℒ_{\infty} ​ (𝕏)} = inf {a \geq 0 : λ ​ ({x \in 𝕏 : | f ​ (x) | > a}) = 0} < \infty,

where we call $∥ \cdot ∥_{ℒ_{p} (𝕏)}$ the $ℒ_{p} -norm$ on $𝕏$ . When $𝕏 = ℝ^{n}$ , we shall write $∥ \cdot ∥_{ℒ_{p} (ℝ^{n})} = ∥ \cdot ∥_{ℒ_{p}}$ . In addition, we define the so-called Kullback-Leibler divergence, see [17], between any two pdfs $f$ and $g$ on $𝕏$ as

{KL}_{𝕏} ​ (f, g) = \int 𝟏_{𝕏} ​ f ​ \log (\frac{f}{g}) ​ d ​ λ ​ .

In [28], the approximation of pdfs $f$ by the class $ℳ_{m}^{g}$ was explored in a restrictive setting. Let ${h_{m}^{g}}$ be a sequence of functions that draw elements from the nested sequence of sets ${ℳ_{m}^{g}}$ (i.e., $h_{1}^{g} \in ℳ_{1}^{g}, h_{2}^{g} \in ℳ_{2}^{g}, \dots$ ). The following result of [40] was presented in [28], along with a collection of its implications, such as the results of from [19] and [30].

Theorem 1 (Zeevi and Meir, 1997).

If

f \in {f : 𝟏_{𝕂} ​ f \geq β ​, ​ β > 0} \cap ℒ_{2} ​ (𝕂)

and $g$ are pdfs and $𝕂$ is compact, then there exists a sequence ${h_{m}^{g}}$ such that

lim_{m \to \infty} {‖ f - h_{m}^{g} ‖}_{ℒ_{2} ​ (𝕂)} = 0 ​ and ​ lim_{m \to \infty} {KL}_{𝕂} ​ (f, h_{m}^{g}) = 0 ​ .

Although powerful, this result is restrictive in the sense that it only permits approximation in the $ℒ_{2}$ norm on compact sets $𝕂$ , and that the result only allows for approximation of functions $f$ that are strictly positive on $𝕂$ . In general, other modes of approximation are desirable, in particular approximation in $ℒ_{p} -norm$ for $p = 1$ or $p = \infty$ are of interest, where the latter case is generally referred to as uniform approximation. Furthermore, the strict-positivity assumption, and the restriction on compact sets limits the scope of applicability of Theorem 1. An example of an interesting application of extensions beyond Theorem 1 is within the $ℒ_{1} -norm$ approximation framework of [8].

Let $g : ℝ^{n} \to ℝ$ again be a pdf. Then, for each $m \in ℕ$ , we define

𝒩_{m}^{g}

= {h : h ​ (x) = \sum_{i = 1}^{m} c_{i} ​ \frac{1}{σ_{i}^{n}} ​ g ​ (\frac{x - μ_{i}}{σ_{i}}) ​, ​ μ_{i} \in ℝ^{n} ​, ​ σ_{i} \in ℝ_{+} ​, ​ c_{i} \in ℝ ​, ​ i \in [m]} ​,

which we call the set of $m -component$ location-scale linear combinations of the pdf $g$ . In the past, results regarding approximations of pdfs $f$ via functions $η \in 𝒩_{m}^{g}$ have been more forthcoming. For example, in the case of $g = ϕ$ , where

ϕ ​ (x) = {(2 ​ π)}^{- n / 2} ​ \exp (- {‖ x ‖}_{2}^{2} / 2) ​,

(1)

is the standard normal pdf. Denoting the class of continuous functions with support on $ℝ^{n}$ by $𝒞$ . We have the result that for every pdf $f$ , compact set $𝕂 \subset ℝ^{n}$ , and $ϵ > 0$ , there exists an $m \in ℕ$ and $h \in 𝒩_{m}^{ϕ}$ , such that ${‖ f - h ‖}_{ℒ_{\infty} (𝕂)} < ϵ$ [32, Lem. 1]. Furthermore, upon defining the set of continuous functions that vanish at infinity by

𝒞_{0}

= {f \in 𝒞 : \forall ϵ > 0, \exists a compact ​ 𝕂 \subset ℝ^{n} ​, such that ​ {‖ f ‖}_{ℒ_{\infty} ​ (𝕂^{∁})} < ϵ} ​,

we also have the result: for every pdf $f \in 𝒞_{0}$ and $ϵ > 0$ , there exists an $m \in ℕ$ and $h \in 𝒩_{m}^{ϕ}$ , such that ${‖ f - h ‖}_{ℒ_{\infty}} < ϵ$ [32, Thm. 2]. Both of the results from [32] are simple implications of the famous Stone-Weierstrass theorem (cf. [34] and [7]).

To the best of our knowledge, the strongest available claim that is made regarding the folk theorem, within a probabilistic or statistical context, is that of [6, Thm. 33.2]. Let ${η_{m}^{g}}$ be a sequence of functions that draw elements from the nested sequence of sets ${𝒩_{m}^{g}}$ , in the same manner as ${h_{m}^{g}}$ . We paraphrase the claim without loss of fidelity, as follows.

Claim 1.

If $f, g \in 𝒞$ are pdfs and $𝕂 \subset ℝ^{n}$ is compact, then there exists a sequence ${η_{m}^{g}}$ , such that

lim_{m \to \infty} {‖ f - η_{m}^{g} ‖}_{ℒ_{\infty} ​ (𝕂)} = 0 ​ .

Unfortunately, the proof of Claim 1 is not provided within [6]. The only reference of the result is to an undisclosed location in [4], which, upon investigation, can be inferred to be Theorem 5 of [4, Ch. 20]. It is further notable that there is no proof provided for the theorem. Instead, it is stated that the proof is similar to that of Theorem 1 in [4, Ch. 24], which is a reproduction of the proof for [38, Lem. 3.1].

There is a major problem in applying the proof technique of [38, Lem. 3.1] in order to prove Claim 1. The proof of [38, Lem. 3.1] critically depends upon the statement that “there is no loss of generality in assuming that $f (x) = 0$ for $x \in ℝ^{n} \ 2 𝕂$ ”. Here, for $a \in ℝ_{+}$ , $a 𝕂 = {x \in ℝ^{n} : x = a y , y \in 𝕂}$ . The assumption is necessary in order to write any convolution with $f$ and an arbitrary continuous function as an integral over a compact domain, and then to use a Riemann sum to approximate such an integral. Subsequently, such a proof technique does not work outside the class of continuous functions that are compactly supported on $a 𝕂$ . Thus, one cannot verify Claim 1 from the materials of [38], [4], and [6], alone.

Some recent results in the spirit of Claim 1 have been obtained by [27] and [26], using methods from the study of universal series (see for example in [25]).

Let

𝒲 = {f \in 𝒞_{0} : \sum_{y \in ℤ^{n}} sup_{x \in {[0, 1]}^{n}} ​ | f ​ (x + y) | < \infty}

denote the so-called Wiener’s algebra (see, e.g., [11]) and let

𝒱

= {f \in 𝒞_{0} : \forall x \in ℝ^{n} ​, ​ | f ​ (x) | \leq β ​ {(1 + {‖ x ‖}_{2})}^{- n - θ} ​, ​ β, θ \in ℝ_{+}}

be a class of functions with tails decaying at a faster rate than $o ({‖ x ‖}_{2}^{n})$ . In [26], it is noted that $𝒱 \subset 𝒲$ . Further, let

𝒞_{c} = {f \in 𝒞 : \exists a compact set ​ 𝕂 ​, such that ​ 𝟏_{𝕂^{∁}} ​ f = 0} ​,

denote the set of compactly supported continuous functions. The following theorem was proved in [27].

Theorem 2 (Nestoridis and Stefanopoulos, 2007, Thm. 3.2).

If $g \in 𝒱$ , then the following statements hold.

(a)

For any $f \in 𝒞_{c}$ , there exists a sequence ${η_{m}^{g}}$ ( $η_{m}^{g} \in 𝒩_{m}^{g}$ ), such that

$lim_{m \to \infty} {‖ f - η_{m}^{g} ‖}_{ℒ_{1}} + {‖ f - η_{m}^{g} ‖}_{ℒ_{\infty}} = 0 .$
(b)

For any $f \in 𝒞_{0}$ , there exists a sequence ${η_{m}^{g}}$ ( $η_{m}^{g} \in 𝒩_{m}^{g}$ ), such that

$lim_{m \to \infty} {‖ f - η_{m}^{g} ‖}_{ℒ_{\infty}} = 0 .$
(c)

For any $1 \leq p < \infty$ and $f \in ℒ_{p}$ , there exists a sequence ${η_{m}^{g}}$ ( $η_{m}^{g} \in 𝒩_{m}^{g}$ ), such that

$lim_{m \to \infty} {‖ f - η_{m}^{g} ‖}_{ℒ_{p}} = 0 .$
(d)

For any measurable $f$ , there exists a sequence ${η_{m}^{g}}$ ( $η_{m}^{g} \in 𝒩_{m}^{g}$ ), such that

$lim_{m \to \infty} η_{m}^{g} = f , almost everywhere.$
(e)

If $ν$ is a $σ -finite$ Borel measure on $ℝ^{n}$ , then for any $ν -measurable$ $f$ , there exists a sequence ${η_{m}^{g}}$ ( $η_{m}^{g} \in 𝒩_{m}^{g}$ ), such that

$lim_{m \to \infty} η_{m}^{g} = f,$

almost everywhere, with respect to $ν$ .

The result was then improved upon, in [26], whereupon the more general space $𝒲$ was taken as a replacement for $𝒱$ , in Theorem 2. Denote the class of bounded continuous functions by $𝒞_{b} = 𝒞 \cap ℒ_{\infty}$ . The following theorem was proved in [26].

Theorem 3 (Nestoridis et al., 2011, Thm. 3.2).

If $g \in 𝒲$ , then the following statements are true.

(a)

The conclusion of Theorem 2(a) holds, with $𝒞_{c}$ replaced by $𝒞_{0} \cap ℒ_{1}$ .
(b)

The conclusions of Theorem 2(b)–(e) hold.
(c)

For any $f \in 𝒞_{b}$ and compact $𝕂 \subset ℝ^{n}$ , there exists a sequence ${η_{m}^{g}}$ , such that

$lim_{m \to \infty} {‖ f - η_{m}^{g} ‖}_{ℒ_{\infty} (𝕂)} = 0 .$

Utilizing the techniques from [27], [1] proved a similar set of results to Theorem 2, under the restriction that $f$ is a non-negative function with support $ℝ$ , using $g = ϕ$ (i.e. $g$ has form (1), where $n = 1$ ) and taking ${h_{m}^{ϕ}}$ as the approximating sequence, instead of ${η_{m}^{g}}$ . That is, the following result is obtained.

Theorem 4 (Bacharoglou, 2010, Cor. 2.5).

If $f : ℝ \to ℝ_{+} \cup {0}$ , then the following statements are true.

(a)

For any pdf $f \in 𝒞_{c}$ , there exists a sequence ${h_{m}^{ϕ}}$ ( $h_{m}^{ϕ} \in ℳ_{m}^{ϕ}$ ), such that

$lim_{m \to \infty} {‖ f - h_{m}^{ϕ} ‖}_{ℒ_{1}} + {‖ f - h_{m}^{ϕ} ‖}_{ℒ_{\infty}} = 0 .$
(b)

For any $f \in 𝒞_{0}$ , such that ${‖ f ‖}_{ℒ_{1}} \leq 1$ , there exists a sequence ${h_{m}^{ϕ}}$ ( $h_{m}^{ϕ} \in ℳ_{m}^{ϕ}$ ), such that

$lim_{m \to \infty} {‖ f - h_{m}^{ϕ} ‖}_{ℒ_{\infty}} = 0 .$
(c)

For any $1 < p < \infty$ and $f \in 𝒞 \cap ℒ_{p}$ , such that ${‖ f ‖}_{ℒ_{1}} \leq 1$ , there exists a sequence ${h_{m}^{ϕ}}$ ( $h_{m}^{ϕ} \in ℳ_{m}^{ϕ}$ ), such that

$lim_{m \to \infty} {‖ f - h_{m}^{ϕ} ‖}_{ℒ_{p}} = 0 .$
(d)

For any measurable $f$ , there exists a sequence ${h_{m}^{ϕ}}$ ( $h_{m}^{ϕ} \in ℳ_{m}^{ϕ}$ ), such that

$lim_{m \to \infty} h_{m}^{ϕ} = f , almost everywhere.$
(e)

For any pdf $f \in 𝒞$ , there exists a sequence ${h_{m}^{ϕ}}$ ( $h_{m}^{ϕ} \in ℳ_{m}^{ϕ}$ ), such that

$lim_{m \to \infty} {‖ f - h_{m}^{ϕ} ‖}_{ℒ_{1}} = 0 .$

To the best of our knowledge, Theorem 4 is the most complete characterization of the approximating capabilities of the mixture of normal distributions. However, it is restrictive in two ways. First, it does not permit characterization of approximation via the class $ℳ_{m}^{g}$ for any $g$ except the normal pdf $ϕ$ . Although $ϕ$ is traditionally the most common choice for $g$ in practice, the modern mixture model literature has seen the use of many more exotic component pdfs, such as the student-t pdf and its skew and modified variants (see, e.g., [29], [13], and [18]). Thus, its use is somewhat limited in the modern context. Furthermore, modern applications tend to call for $n > 1$ , further restricting the impact of the result as a theoretical bulwark for finite mixture modeling in practice. A remark in [1] states that the result can generalized to the case where $g \in 𝒱$ instead of $g = ϕ$ . However, no suggestions were proposed, regarding the generalization of Theorem 4 to the case of $n > 1$ .

In this article, we prove a novel set of results that largely generalize Theorem 4. Using techniques inspired by [9] and [4], we are able to obtain a set of results regarding the approximation capability of the class of $m -component$ mixture models $ℳ_{m}^{g}$ , when $g \in 𝒞_{0}$ or $g \in 𝒱$ , and for any $n \in ℕ$ . By definition of $𝒱$ , the majority of our results extend beyond the proposed possible generalizations of Theorem 4.

The article proceeds as follows. Our main theorem is stated and its seperate parts are proved in the Section 2. Comments and discussion are provided in Section 3. Necessary technical lemmas and results are also included, for reference, in the Appendix.

2 Main result

The remainder of the article is devoted to proving the following theorem.

Theorem 5 (Main result).

If we assume that $f$ and $g$ are pdfs and that $g \in 𝒞_{0}$ , then the following statements are true.

(a)

For any $f \in 𝒞_{0}$ , there exists a sequence ${h_{m}^{g}}$ ( $h_{m}^{g} \in ℳ_{m}^{g}$ ), such that

$lim_{m \to \infty} {‖ f - h_{m}^{g} ‖}_{ℒ_{\infty}} = 0 .$
(b)

For any $f \in 𝒞_{b}$ and compact $𝕂 \subset ℝ^{n}$ , there exists a sequence ${h_{m}^{g}}$ ( $h_{m}^{g} \in ℳ_{m}^{g}$ ), such that

$lim_{m \to \infty} {‖ f - h_{m}^{g} ‖}_{ℒ_{\infty} (𝕂)} = 0 .$
(c)

For any $1 < p < \infty$ and $f \in ℒ_{p}$ , there exists a sequence ${h_{m}^{g}}$ ( $h_{m}^{g} \in ℳ_{m}^{g}$ ), such that

$lim_{m \to \infty} {‖ f - h_{m}^{g} ‖}_{ℒ_{p}} = 0 .$
(d)

For any measurable $f$ , there exists a sequence ${h_{m}^{g}}$ ( $h_{m}^{g} \in ℳ_{m}^{g}$ ), such that

$lim_{m \to \infty} h_{m}^{g} = f , almost everywhere.$
(e)

If $ν$ is a $σ -finite$ Borel measure on $ℝ^{n}$ , then for any $ν -measurable$ $f$ , there exists a sequence ${h_{m}^{g}}$ ( $h_{m}^{g} \in ℳ_{m}^{g}$ ), such that

$lim_{m \to \infty} h_{m}^{g} = f,$

almost everywhere, with respect to $ν$ .

If we assume instead that $g \in 𝒱$ , then the following statement is also true.

(f)

For any $f \in 𝒞$ , there exists a sequence ${h_{m}^{g}}$ ( $h_{m}^{g} \in ℳ_{m}^{g}$ ), such that

$lim_{m \to \infty} {‖ f - h_{m}^{g} ‖}_{ℒ_{1}} = 0 .$

2.1 Technical preliminaries

Before we begin to prove the main theorem, we establish some technical results regarding our class of component densities $𝒞_{0}$ . Let $f, g \in ℒ_{1}$ and denote the convolution of $f$ and $g$ by $f ⋆ g = g ⋆ f$ . Further, we denote the sequence of dilates of $g$ by ${g_{k} : g_{k} (x) = k^{n} g (k x), k \in ℕ} .$ The following result is an alternative to Lemma 5 and Corollary 1. Here, we replace a boundedness assumption on the approximand, in the aforementioned theorem by a vanishing at infinity assumption, instead.

Lemma 1.

Let $g$ be a pdf and $f \in 𝒞_{0}$ , such that ${‖ f ‖}_{ℒ_{\infty}} > 0$ . Then,

lim_{k \to \infty} {‖ g_{k} ⋆ f - f ‖}_{ℒ_{\infty}} = 0 ​ .

Proof.

It suffices to show that for any $ϵ > 0$ , there exists a $k (ϵ) \in ℕ$ , such that ${‖ g_{k} ⋆ f - f ‖}_{ℒ_{\infty}} < ϵ$ , for all $k \geq k (ϵ)$ . By Lemma 6, $f \in 𝒞_{b}$ , and thus ${‖ f ‖}_{ℒ_{\infty}} < \infty$ . By making the substitution $z = k x$ , we obtain for each $k$

\int g_{k} ​ (x) ​ d ​ λ = \int k^{n} ​ g ​ (k ​ x) ​ d ​ λ = \int g ​ (z) ​ d ​ λ = 1 ​ .

By Corollary 1, we obtain ${lim}_{k \to \infty} \int 𝟏_{{x : {‖ x ‖}_{2} > δ}} g_{k} d λ = 0$ and thus we can choose a $k (ϵ)$ , such that

\int 𝟏_{{x : {‖ x ‖}_{2} > δ}} ​ g_{k} ​ d ​ λ < \frac{ϵ}{4 ​ {‖ f ‖}_{ℒ_{\infty}}} ​ .

Since $g$ is a pdf, we have

	$\| (g_{k} ⋆ f) (x) - f (x) \|$	$= \| \int g_{k} (y) [f (x - y) - f (x)] d λ (y) \|$
		$\leq \int g_{k} (y) \| f (x - y) - f (x) \| d λ (y) .$

By uniform continuity, for any $ϵ > 0$ , there exists a $δ (ϵ) > 0$ such that $| f (x - y) - f (x) | < ϵ / 2$ , for any $x, y \in ℝ^{n}$ , such that ${‖ y ‖}_{2} < δ (ϵ)$ (Lemma 6). Thus, on the one hand, for any $δ (ϵ)$ , we can pick a $k (ϵ)$ such that

	$\int 𝟏_{{y : {‖ y ‖}_{2} > δ (ϵ)}} g_{k} (y) \| f (x - y) - f (x) \| d λ (y)$
	$\leq 2 {‖ f ‖}_{ℒ_{\infty}} \int 𝟏_{{y : {‖ y ‖}_{2} > δ (ϵ)}} g_{k} d λ$
	$\leq 2 {‖ f ‖}_{ℒ_{\infty}} \times \frac{ϵ}{4 {‖ f ‖}_{ℒ_{\infty}}} = \frac{ϵ}{2} ,$		(2)

and on the other hand

	$\int 𝟏_{{y : {‖ y ‖}_{2} \leq δ (ϵ)}} g_{k} (y) \| f (x - y) - f (x) \| d λ (y)$
	$\leq \frac{ϵ}{2} \int 𝟏_{{y : {‖ y ‖}_{2} \leq δ (ϵ)}} g_{k} d λ$
	$\leq \frac{ϵ}{2} \times 1 = \frac{ϵ}{2} .$		(3)

The proof is completed by summing (2) and (3). ∎

Lemma 2.

If $f \in 𝒞_{0}$ is such that $f \geq 0$ , and $ϵ > 0$ , then there exists a $h \in 𝒞_{c}$ , such that $0 \leq h \leq f$ , and

{‖ f - h ‖}_{ℒ_{\infty}} < ϵ

Proof.

Since $f \in 𝒞_{0}$ , there exists a compact $𝕂 \subset ℝ^{n}$ such that ${‖ f ‖}_{ℒ_{\infty} (𝕂^{∁})} < ϵ / 2$ . By Lemma 7, there exists some $g \in 𝒞_{c}$ , such that $0 \leq g \leq 1$ and $𝟏_{𝕂} g = 1$ . Let $h = g f$ , which implies that $h \geq 0$ and $0 \leq h \leq f$ . Furthermore, notice that $𝟏_{𝕂} (f - h) = 0$ and ${‖ h ‖}_{ℒ_{\infty}} \leq {‖ f ‖}_{ℒ_{\infty}}$ , by construction. The proof is completed by observing that

	${‖ f - h ‖}_{ℒ_{\infty}}$	$= {‖ f - h ‖}_{ℒ_{\infty} (𝕂^{∁})}$
		$\leq {‖ f ‖}_{ℒ_{\infty} (𝕂^{∁})} + {‖ h ‖}_{ℒ_{\infty} (𝕂^{∁})}$
		$\leq 2 {‖ f ‖}_{ℒ_{\infty} (𝕂^{∁})} < ϵ .$

∎

For any $δ > 0$ , uniformly continuous function $f$ , let

w ​ (f, δ) = sup_{{x, y \in ℝ^{n} : {‖ x - y ‖}_{2} \leq δ}} | f ​ (x) - f ​ (y) |

denote the modulus of continuity of $f$ . Furthermore, define the diameter of a set $𝕏 \subset ℝ^{n}$ by $diam (𝕏) = {sup}_{x, y \in 𝕏} {‖ x - y ‖}_{2}$ and denote an open ball, centered at $x \in ℝ^{n}$ with radius $r > 0$ by $𝔹 (x, r) = {y \in ℝ^{n} : {‖ x - y ‖}_{2} < r}$ .

Notice that the class $ℳ_{m}^{g}$ can be parameterized as

	$ℳ_{m}^{g}$	$= {h : h (x) = \sum_{i = 1}^{m} c_{i} k_{i}^{n} g (k_{i} x - z_{i}),$
		$z_{i} \in ℝ^{n}, k_{i} \in ℝ_{+}, c \in 𝕊^{m - 1}, i \in [m]},$

where $k_{i} = 1 / σ_{i}$ and $z_{i} = μ_{i} / σ_{i}$ . The following result is the primary mechanism that permits us to construct finite mixture approximations for convolutions of form $g_{k} ⋆ f$ . The argument motivated by the approaches taken in Theorem 1 in [4, Ch. 24], [27, Lem. 3.1], and [26, Thm. 3.1].

Lemma 3.

Let $f \in 𝒞$ and $g \in 𝒞_{0}$ be pdfs. Furthermore, let $𝕂 \subset ℝ^{n}$ be compact and $h \in 𝒞_{c}$ , where $𝟏_{𝕂^{∁}} h = 0$ and $0 \leq h \leq f$ . Then for any $k \in ℕ$ , there exists a sequence ${h_{m}^{g}}$ , such that

lim_{m \to \infty} {‖ g_{k} ⋆ h - h_{m}^{g} ‖}_{ℒ_{\infty}} = 0 .

Proof.

It suffices to show that for any $k \in ℕ$ and $ϵ > 0$ , there exists a sufficiently large enough $m (ϵ) \in ℕ$ so that for all $m \geq m (ϵ), h_{m}^{g} \in ℳ_{m}^{g}$ such that

{‖ g_{k} ⋆ h - h_{m}^{g} ‖}_{ℒ_{\infty}} < ϵ .

(4)

For any $k \in ℕ$ , we can write

	$(g_{k} ⋆ h) (x)$	$= \int g_{k} (x - y) h (y) d λ (y)$
		$= \int 𝟏_{{y : y \in 𝕂}} g_{k} (x - y) h (y) d λ (y)$
		$= \int 𝟏_{{y : y \in 𝕂}} k^{n} g (k x - k y) h (y) d λ (y)$
		$= \int 𝟏_{{z : z \in k 𝕂}} g (k x - z) h (\frac{z}{k}) d λ (z) .$

Here, $k 𝕂$ is continuous image of a compact set, and hence is compact (cf. [31, Thm. 4.14]). By Lemma 8, for any $δ > 0$ , there exists $κ_{i} \in ℝ^{n}$ ( $i \in [m - 1]$ , $m \in ℕ$ ), such that $k 𝕂 \subset ⋃_{i = 1}^{m - 1} 𝔹 (κ_{i}, δ / 2)$ . Further, if $𝔹_{i}^{δ} = k 𝕂 \cap 𝔹 (κ_{i}, δ / 2)$ , then we have $k 𝕂 = ⋃_{i = 1}^{m - 1} 𝔹_{i}^{δ}$ . We can obtain a disjoint covering of $k 𝕂$ by taking $𝔸_{1}^{δ} = 𝔹_{1}$ and $𝔸_{i}^{δ} = 𝔹_{i}^{δ} \ ⋃_{j = 1}^{i - 1} 𝔹_{j}^{δ}$ ( $i \in [m - 1]$ ) and noting that $k 𝕂 = ⋃_{i = 1}^{m - 1} 𝔸_{i}^{δ}$ , by construction (cf. [4, Ch. 24]). Furthermore, each $𝔸_{i}^{δ}$ is a Borel set and $diam (𝔸_{i}^{δ}) \leq δ$ .

For convenience, let $Π_{m}^{δ} = {𝔸_{i}^{δ} : i \in [m - 1]}$ denote the disjoint covering, or partition, of $k 𝕂$ . We seek to show that there exists an $m \in ℕ$ and $Π_{m}^{δ}$ , such that

{‖ g_{k} ⋆ h - \sum_{i = 1}^{m} c_{i} ​ k_{i}^{n} ​ g ​ (k_{i} ​ x - z_{i}) ‖}_{ℒ_{\infty}} < ϵ ​,

where $k_{i} = k$ ,

c_{i} = k^{- n} ​ \int 𝟏_{{z : z \in 𝔸_{i}^{δ}}} ​ h ​ (z / k) ​ d ​ λ ​ (z),

and $z_{i} \in 𝔸_{i}^{δ}$ , for $i \in [m - 1]$ .

Further, $z_{m} \in 𝔸_{m - 1}^{δ}$ and $c_{m} = 1 - \sum_{i = 1}^{m - 1} c_{i}$ , with $k_{m}$ chosen as follows. By Lemma 6, $g \leq C < \infty$ for some positive $C$ . Then, ${‖ c_{m} k_{m}^{n} g (k_{m} x - z_{m}) ‖}_{ℒ_{\infty}} \leq c_{m} k_{m}^{n} C$ . We may choose $k_{m}$ so that $k_{m}^{n} = ϵ / (2 c_{m} C)$ , so that

{‖ c_{m} ​ k_{m}^{n} ​ g ​ (k_{m} ​ x - z_{m}) ‖}_{ℒ_{\infty}} \leq \frac{ϵ}{2} ​ .

Since $0 \leq h \leq f$ , the sum of $c_{i}$ ( $i \in [m - 1]$ ) satisfies the inequality

	$\sum_{i = 1}^{m - 1} c_{i}$	$= k^{- n} \sum_{i = 1}^{m - 1} \int 𝟏_{{z : z \in 𝔸_{i}^{δ}}} h (\frac{z}{k}) d λ$
		$= k^{- n} \int 𝟏_{{z : z \in k 𝕂}} h (\frac{z}{k}) d λ$
		$= \int 𝟏_{{x : x \in 𝕂}} h d λ \leq \int 𝟏_{{x : x \in 𝕂}} f d λ \leq \int f d λ = 1 .$

Thus, $0 \leq c_{m} \leq 1$ , and our construction implies that $h_{m}^{g} \in ℳ_{m}^{g} ,$ where

h_{m}^{g} ​ (x) = \sum_{i = 1}^{m} c_{i} ​ k_{i}^{n} ​ g ​ (k_{i} ​ x - z_{i}) ​ \forall x \in ℝ^{n} ​ .

We can bound the left-hand side of (4) as follows:

		${‖ g_{k} ⋆ h - h_{g}^{m} ‖}_{ℒ_{\infty}}$
		$\leq {‖ (g_{k} ⋆ h) (x) - \sum_{i = 1}^{m - 1} c_{i} k_{i}^{n} g (k_{i} x - z_{i}) ‖}_{ℒ_{\infty}}$
		$+ {‖ c_{m} k_{m}^{n} g (k_{m} x - z_{m}) ‖}_{ℒ_{\infty}}$
		$\leq {‖ (g_{k} ⋆ h) (x) - \sum_{i = 1}^{m - 1} c_{i} k_{i}^{n} g (k_{i} x - z_{i}) ‖}_{ℒ_{\infty}} + \frac{ϵ}{2}$
		$= ∥ \int 𝟏_{{z : z \in k 𝕂}} g (k x - z) h (\frac{z}{k}) d λ (z)$
		$- {\sum_{i = 1}^{m - 1} \int 𝟏_{{z : z \in 𝔸_{i}^{δ}}} g (k x - z_{i}) h (\frac{z}{k}) d λ (z) ∥}_{ℒ_{\infty}} + \frac{ϵ}{2}$
		$\leq \sum_{i = 1}^{m - 1} \int 𝟏_{{z : z \in 𝔸_{i}^{δ}}} {‖ g (k x - z) - g (k x - z_{i}) ‖}_{ℒ_{\infty}} h (\frac{z}{k}) d λ (z) + \frac{ϵ}{2} .$		(5)

Since

{‖ k ​ x - z - (k ​ x - z_{i}) ‖}_{2} = {‖ z - z_{i} ‖}_{2} \leq diam ​ (𝔸_{i}^{δ}) \leq δ ​,

we have $| g (k x - z) - g (k x - z_{i}) | \leq w (g, δ)$ , for each $i \in [m - 1]$ . Since ${lim}_{δ \to 0} w (g, δ) = 0$ (cf. [21, Thm. 4.7.3]), we may choose a $δ (ϵ) > 0$ so that $w (g, δ (ϵ)) < ϵ / (2 k^{n})$ . We may proceed from (2.1) as follows:

${‖ g_{k} ⋆ h - h_{g}^{m} ‖}_{ℒ_{\infty}}$	$\leq w (g, δ (ϵ)) \int 𝟏_{{z : z \in k 𝕂}} h (\frac{z}{k}) d λ + \frac{ϵ}{2}$
	$= w (g, δ (ϵ)) k^{n} \int h d λ + \frac{ϵ}{2}$
	$\leq w (g, δ (ϵ)) k^{n} + \frac{ϵ}{2}$
	$< \frac{ϵ}{2} + \frac{ϵ}{2} = ϵ .$	(6)

To conclude the proof, it suffices to choose an appropriate sequence of partitions $Π_{m}^{δ (ϵ)}, m \geq m (ϵ)$ , for some large but finite $m (ϵ)$ , so that (2.1) and (6) hold, which is possible by Lemma 8. ∎

For any $r \in ℕ$ , let ${\bar{𝔹}}_{r} = {x \in ℝ^{n} : {‖ x ‖}_{2} \leq r}$ be a closed ball of radius $r$ , centered at the origin.

Lemma 4.

If $f \in ℒ_{1}$ , such that $f \geq 0$ , then

lim_{r \to \infty} {‖ f - 𝟏_{{\bar{𝔹}}_{r}} ​ f ‖}_{ℒ_{1}} = 0 ​ .

Proof.

By construction, each element of the sequence ${𝟏_{{\bar{𝔹}}_{r}} f}$ ( $r \in ℕ$ ) is measurable, $0 \leq 𝟏_{{\bar{𝔹}}_{r}} f \leq f$ , and

lim_{r \to \infty} 𝟏_{{\bar{𝔹}}_{r}} ​ f = f ​,

point-wise. We obtain our conclusion via the Lesbegue dominated convergence theorem. ∎

2.2 Proof of Theorem 5(a)

We now proceed to prove each of the parts of Theorem 5. To prove Theorem 5(a) it suffices to show that for every $ϵ > 0$ , there exists a $h_{m}^{g} \in ℳ_{m}^{g}$ , such that ${‖ f - h_{m}^{g} ‖}_{ℒ_{\infty}} < ϵ .$

Start by applying Lemma 2 to obtain $h \in 𝒞_{c}$ , such that $0 \leq h \leq f$ and ${‖ f - h ‖}_{ℒ_{\infty}} < ϵ / 2$ . Then, we have

	${‖ f - h_{m}^{g} ‖}_{ℒ_{\infty}}$	$\leq {‖ f - h ‖}_{ℒ_{\infty}} + {‖ h - h_{m}^{g} ‖}_{ℒ_{\infty}}$
		$< \frac{ϵ}{2} + ∥ h - {h_{m}^{g} ∥}_{ℒ_{\infty}} .$		(7)

The goal is to find a $h_{m}^{g}$ , such that ${‖ h - h_{m}^{g} ‖}_{ℒ_{\infty}} < ϵ / 2$ . Since $h \in 𝒞_{c}$ , we may find a compact $𝕂 \subset ℝ^{n}$ such that ${‖ h ‖}_{ℒ_{\infty} (𝕂^{∁})} = 0$ . Apply Lemma 1 to show the existence of a $k (ϵ)$ , such that

{‖ h - g_{k} ⋆ h ‖}_{ℒ_{\infty}} < \frac{ϵ}{4} ​,

for all $k \geq k (ϵ)$ . With a fixed $k = k (ϵ)$ , apply Lemma 3 to show that there exists a $h_{m}^{g} \in ℳ_{m}^{g}$ , such that

{‖ g_{k ​ (ϵ)} ⋆ h - h_{m}^{g} ‖}_{ℒ_{\infty}} < \frac{ϵ}{4} ​ .

By the triangle inequality, we have

	${‖ h - h_{m}^{g} ‖}_{ℒ_{\infty}}$	$\leq {‖ h - g_{k (ϵ)} ⋆ h ‖}_{ℒ_{\infty}} + {‖ g_{k (ϵ)} ⋆ h - h_{m}^{g} ‖}_{ℒ_{\infty}}$
		$< \frac{ϵ}{4} + \frac{ϵ}{4} = \frac{ϵ}{2} .$		(8)

The proof is complete by substitution of (8) into (7).

2.3 Proof of Theorem 5(b)

For any $ϵ > 0$ and compact $𝕂 \subset ℝ^{n}$ , it suffices to show that there exists a sufficiently large enough $m (ϵ) \in ℕ$ so that for all $m \geq m (ϵ), h_{m}^{g} \in ℳ_{m}^{g},$ such that ${‖ f - h_{m}^{g} ‖}_{ℒ_{\infty} (𝕂)} < ϵ$ .

By Lemma 5, we can find a $k (ϵ, 𝕂) \in ℕ$ , such that

{‖ f - g_{k} ⋆ f ‖}_{ℒ_{\infty} ​ (𝕂)} < \frac{ϵ}{3} ​,

(9)

for every $k \geq k (ϵ, 𝕂)$ . Since $g \in 𝒞_{0}$ , ${‖ g ‖}_{ℒ_{\infty}} \leq C < \infty$ for some positive $C$ , by Lemma 6. For any $k, r \in ℕ$ , via Young’s convolution inequality:

{‖ g_{k} ⋆ f - g_{k} ⋆ (𝟏_{{\bar{𝔹}}_{r}} ​ f) ‖}_{ℒ_{\infty}} \leq k^{n} ​ C ​ \int (𝟏_{{\bar{𝔹}}_{r}^{∁}} ​ f) ​ d ​ λ = k^{n} ​ C ​ {‖ f - 𝟏_{{\bar{𝔹}}_{r}} ​ f ‖}_{ℒ_{1}} ​ .

(10)

For fixed $k$ , we may choose $r (ϵ, 𝕂) \in ℕ$ , using Lemma 4, so that ${‖ f - 𝟏_{{\bar{𝔹}}_{r}} f ‖}_{ℒ_{1}} \leq ϵ / (3 k^{n} C)$ and thus the final term of (10) is bounded from above by $ϵ / 3$ for all $r \geq r (ϵ, 𝕂)$ . Thus, for $k = k (ϵ, 𝕂)$ and, $r \geq r (ϵ, 𝕂)$

{‖ g_{k ​ (ϵ, 𝕂)} ⋆ f - g_{k ​ (ϵ, 𝕂)} ⋆ (𝟏_{{\bar{𝔹}}_{r ​ (ϵ, 𝕂)}} ​ f) ‖}_{ℒ_{\infty}} \leq \frac{ϵ}{3} ​ .

(11)

Using Lemma 3, with approximand $𝟏_{{\bar{𝔹}}_{r (ϵ, 𝕂)}} f$ , component density $g$ , compact set ${\bar{𝔹}}_{r (ϵ, 𝕂)}$ , $h = 𝟏_{{\bar{𝔹}}_{r (ϵ, 𝕂)}} f$ , and with $k = k (ϵ, 𝕂)$ fixed, we have the existence of a density $h_{m}^{g} \in ℳ_{m}^{g}, m \geq m (ϵ) \in ℕ,$ such that

{‖ g_{k ​ (ϵ, 𝕂)} ⋆ (𝟏_{{\bar{𝔹}}_{r ​ (ϵ, 𝕂)}} ​ f) - h_{m}^{g} ‖}_{ℒ_{\infty}} \leq \frac{ϵ}{3} ​ .

(12)

We obtain the desired result by combining (9), (11), and (12), via the triangle inequality.

2.4 Proof of Theorem 5(c)

The technique used to prove Theorem 5(c) is different to those used in the previous sections. Here, we use a result of [9] that generalizes the classic Barron-Jones Hilbert space approximation result (cf. [16] and [2]) to Banach spaces.

To prove Theorem 5(c), it suffices to show that for every $ϵ > 0$ , there exists a sufficiently large enough $m (ϵ) \in ℕ$ so that for all $m \geq m (ϵ), h_{m}^{g} \in ℳ_{m}^{g}$ such that ${‖ f - h_{m}^{g} ‖}_{ℒ_{p}} < ϵ$ . Begin by applying Corollary 1 to obtain a $k (ϵ)$ , such that

{‖ f - g_{k} ⋆ f ‖}_{ℒ_{p}} < \frac{ϵ}{2}

(13)

for all $k \geq k (ϵ)$ .

For some pdf $g$ and fixed $k \in ℕ$ , let us define the class

𝒢_{g}^{k} = {h : h ​ (x) = k^{n} ​ g ​ (k ​ x - k ​ μ) ​, ​ μ \in ℝ^{n}} ​,

write the $m -point$ convex hull of $𝒢_{g}^{k}$ as

{Conv}_{m} ​ (𝒢_{g}^{k}) = {h : h = \sum_{i = 1}^{m} c_{i} ​ g_{i} ​, ​ g_{i} \in 𝒢_{g}^{k} ​, ​ c \in 𝕊^{m - 1} ​, ​ i \in [m]} ​,

and call ${Conv}_{\infty} (𝒢_{g}^{k}) = Conv (𝒢_{g}^{k})$ the convex hull of $𝒢_{g}^{k}$ . We further say that $\bar{Conv} (𝒢_{g}^{k})$ is the closure of $Conv (𝒢_{g}^{k})$ .

Because $g$ is a pdf, $g \in 𝒞_{0} \subset 𝒞_{b}$ , and $𝒞_{b} \subset ℒ_{\infty}$ , we observe that $g \in ℒ_{1} \cap ℒ_{\infty}$ . Thus, $g \in ℒ_{p}$ , for any $1 < p < \infty$ , by Lemma 9. Since $g$ is a pdf and $f \in ℒ_{p}$ , we have the existence of $g_{k} ⋆ f$ and the fact that ${‖ g_{k} ⋆ f ‖}_{ℒ_{p}}$ is finite.

Furthermore, for any $ψ \in 𝒢_{g}^{k}$ , since $g \in ℒ_{p}$ and by definition of $𝒢_{g}^{k}$ , we have ${‖ ψ ‖}_{ℒ_{p}} \leq k^{n / p} {‖ g ‖}_{ℒ_{p}} .$ Thus, we have

{‖ ψ - g_{k} ⋆ f ‖}_{ℒ_{p}} \leq {‖ ψ ‖}_{ℒ_{p}} + {‖ g_{k} ⋆ f ‖}_{ℒ_{p}} \leq K ​,

(14)

by choosing $K = k^{n / p} {‖ g ‖}_{ℒ_{p}} + {‖ g_{k} ⋆ f ‖}_{ℒ_{p}} > 0$ .

Following [36], we can write the closure of $𝒢_{g}^{k}$ as

\bar{Conv} ​ (G_{g}^{k}) = {h : h ​ (x) = \int k^{n} ​ g ​ (k ​ x - k ​ μ) ​ f ​ (μ) ​ d ​ λ ​ (μ), f ​ is a pdf} ​,

and thus we immediately have $g_{k} ⋆ f \in \bar{Conv} (G_{g}^{k})$ . Combined with (14), we can apply Lemma 11 to obtain the conclusion that there exists a function $h_{m}^{g} \in {Conv}_{m} (𝒢_{g}^{k (ϵ)}) \subset ℳ_{m}^{g}$ , such that

{‖ h_{m}^{g} - g_{k ​ (ϵ)} ⋆ f ‖}_{ℒ_{p}} \leq \frac{K ​ C_{p}}{m^{1 - 1 / α}} ​,

where $α = \min {p, 2}$ and $C_{p}$ is a finite constant. Since $p > 1$ , $m^{1 - 1 / α}$ is strictly increasing, and hence we can choose an $m (ϵ) \in ℕ$ , such that for all $m \geq m (ϵ)$ ,

{‖ h_{m}^{g} - g_{k ​ (ϵ)} ⋆ f ‖}_{ℒ_{p}} \leq \frac{ϵ}{2} ​ .

(15)

The proof is then completed by combining (13) and (15) via the triangle inequality.

2.5 Proof of Theorem 5(d) and Theorem 5(e)

By Theorem 5(a), there exists a sequence ${h_{m}^{g}}$ that uniformly converges to $f$ , as $m \to \infty$ . Thus, by Lemma 12, ${h_{m}^{g}}$ almost uniformly converges to $f$ and also converges almost everywhere, to $f$ , with respect to any measure $ν$ . We prove Theorem 5(d) by setting $ν = λ,$ and we prove Theorem 5(e) by not specifying $ν$ .

2.6 Proof of Theorem 5(f)

It suffices to show that for any $ϵ > 0$ , there exists a sufficiently large enough $m (ϵ) \in ℕ$ so that for all $m \geq m (ϵ), h_{m}^{g} \in ℳ_{m}^{g}$ , where $g \in 𝒱$ , such that ${‖ f - h_{m}^{g} ‖}_{ℒ_{1}} < ϵ$ . Begin by applying Lemma 4 in order to find a $r (ϵ) \in ℕ$ , for any $ϵ > 0$ , such that for all $r \geq r (ϵ)$ ,

{‖ f - 𝟏_{{\bar{𝔹}}_{r}} ​ f ‖}_{ℒ_{1}} \leq \frac{ϵ}{24} < \frac{ϵ}{2} ​,

(16)

where $0 \leq 𝟏_{{\bar{𝔹}}_{r}} f \leq f$ , and $𝟏_{{\bar{𝔹}}_{r}} f \in 𝒞_{c}$ with compact support ${\bar{𝔹}}_{r}$ .

Let $𝕂 = {\bar{𝔹}}_{r}$ and apply the triangle inequality to obtain

	${‖ f - h_{m}^{g} ‖}_{ℒ_{1}}$	$\leq {‖ f - 𝟏_{𝕂} f ‖}_{ℒ_{1}} + {‖ 𝟏_{𝕂} f - h_{m}^{g} ‖}_{ℒ_{1}}$
		$\leq \frac{ϵ}{2} + {‖ 𝟏_{𝕂} f - h_{m}^{g} ‖}_{ℒ_{1}} .$

Hence we need to show that there exists a function $h_{m}^{g} \in ℳ_{m}^{g}$ , such that

{‖ 𝟏_{𝕂} ​ f - h_{m}^{g} ‖}_{ℒ_{1}} \leq \frac{ϵ}{2} ​ .

Since $g \in 𝒱$ and $g_{k} (x) = k^{n} g (k x)$ , by substitution, we have

g_{k} ​ (x) \leq \frac{β ​ k^{- θ}}{{(k^{- 1} + {‖ x ‖}_{2})}^{n + θ}} ​,

(17)

where $β, θ > 0$ are independent of $k$ . By Lemma 5 and Corollary 1, we can obtain a $k_{1} (ϵ)$ , such that for all $k \geq k_{1} (ϵ)$ ,

{‖ 𝟏_{𝕂} ​ f - g_{k} ⋆ (𝟏_{𝕂} ​ f) ‖}_{ℒ_{1}} \leq \frac{ϵ}{4} ​ .

(18)

Suppose that $γ > 1$ and let

𝕂_{k} = {x \in ℝ^{n} : dist ​ (x, 𝕂) \leq k^{- γ}},

where

dist ​ (x, 𝕏) = inf {{‖ x - y ‖}_{2} : y \in 𝕏} ​ .

By construction, $λ (𝕂_{k}) = λ (𝕂) + O (k^{- γ})$ and thus there exists a $k_{2}$ such that $λ (𝕂_{k}) \leq λ (𝕂) + 1$ , for any $k \geq k_{2}$ .

For any $k > k_{2}$ , we can show that

{‖ g_{k} ⋆ (𝟏_{𝕂} ​ f) - h_{m - 1}^{g} ‖}_{ℒ_{1} ​ (𝕂_{k})} < \frac{ϵ}{8} ​ .

(19)

To do so, firstly, for any $x \in ℝ^{n}$ ,

	$g_{k} ⋆ (𝟏_{𝕂} f)$	$= \int 𝟏_{𝕂} g_{k} (x - y) f (y) d λ (y)$
		$= \int 𝟏_{k 𝕂} g (k x - z) f (\frac{z}{k}) d λ (z) .$

To obtain a Riemann sum approximation of $g_{k} ⋆ (𝟏_{𝕂} f)$ , we use an argument analogous to that of Lemma 3. That is, we partition $k 𝕂$ into $m - 1$ disjoint Borel sets $Π_{m} = {𝔸_{1}, \dots, 𝔸_{m - 1}}$ , and we approximate $g_{k} ⋆ (𝟏_{𝕂} f)$ by a $h_{m - 1}^{g} \in ℳ_{m - 1}^{g}$ , where for each $i \in [m - 1]$ , $k_{i} = k$ , $z_{i} \in 𝔸_{i}$ , and

c_{i} = k^{- n} ​ \int 𝟏_{𝔸_{i}} ​ f ​ (\frac{z}{k}) ​ d ​ λ ​ (z) ​ .

Define $k_{m} \in ℝ_{+}$ , $z_{m} \in ℝ^{n}$ , and $c_{m} = 1 - \sum_{i = 1}^{m - 1} c_{i}$ , where

c_{m} = \int f ​ d ​ λ - \int 𝟏_{𝕂} ​ f ​ d ​ λ = {‖ f - 𝟏_{𝕂} ​ f ‖}_{ℒ_{1}} \leq \frac{ϵ}{24}

(20)

by (16). Then, by a similar argument to Lemma 3, $c_{i} \geq 0$ for all $i \in [m]$ and $\sum_{i = 1}^{m} c_{i} = 1$ . Thus, we may define an element $h_{m}^{g} \in ℳ_{m}^{g}$ via the parameters above.

For sufficiently large $k \geq k_{2}$ , we use Lemma 3 to show that

{‖ g_{k} ⋆ (𝟏_{𝕂} ​ f) - h_{m - 1}^{g} ‖}_{ℒ_{\infty} ​ (𝕂_{k})} < \frac{ϵ}{8 ​ (λ ​ (𝕂) + 1)} ​,

which implies

	${‖ g_{k} ⋆ (𝟏_{𝕂} f) - h_{m - 1}^{g} ‖}_{ℒ_{1} (𝕂_{k})}$	$< \int 𝟏_{𝕂_{k}} \frac{ϵ}{8 (λ (𝕂) + 1)} d λ$
		$< \frac{ϵ λ (𝕂_{k})}{8 (λ (𝕂) + 1)} < \frac{ϵ}{8} ,$		(21)

and thus (19) is proved. Using (19), we write

			${‖ g_{k} ⋆ (𝟏_{𝕂} f) - h_{m}^{g} ‖}_{ℒ_{1}}$
		$=$	${‖ g_{k} ⋆ (𝟏_{𝕂} f) - h_{m - 1}^{g} - c_{m} k_{m}^{n} g (k_{m} x - z_{m}) ‖}_{ℒ_{1}}$
		$\leq$	${‖ g_{k} ⋆ (𝟏_{𝕂} f) - h_{m - 1}^{g} ‖}_{ℒ_{1} (𝕂_{k})}$
			$+ {‖ g_{k} ⋆ (𝟏_{𝕂} f) - h_{m - 1}^{g} ‖}_{ℒ_{1} (𝕂_{k}^{∁})}$
			$+ {‖ c_{m} k_{m}^{n} g (k_{m} x - z_{m}) ‖}_{ℒ_{1}}$
		$\leq$	$\frac{ϵ}{8} + c_{m} + {‖ g_{k} ⋆ (𝟏_{𝕂} f) ‖}_{ℒ_{1} (𝕂_{k}^{∁})} + {‖ h_{m - 1}^{g} ‖}_{ℒ_{1} (𝕂_{k}^{∁})} ,$

where ${‖ c_{m} k_{m}^{n} g (k_{m} x - z_{m}) ‖}_{ℒ_{1}} \leq c_{m}$ since $k_{m}^{n} g (k_{m} x - z_{m})$ is a pdf. The aim is now to prove that

{{‖ g_{k} ⋆ (𝟏_{𝕂} ​ f) ‖}_{ℒ_{1} ​ (𝕂_{k}^{∁})} ​ < \frac{ϵ}{24} ​ and ∥ ​ h_{m - 1}^{g} ∥}_{ℒ_{1} ​ (𝕂_{k}^{∁})} < \frac{ϵ}{24} ​ .

Using polar coordinates and (17), we have

	$\int 𝟏_{{x : {‖ x - y ‖}_{2} > k^{- γ}}} g_{k} (x - y) d λ (x)$
	$\leq \int \frac{𝟏_{{x : {‖ x - y ‖}_{2} > k^{- γ}}} β k^{- θ}}{{(k^{- 1} + {‖ x - y ‖}_{2})}^{n + θ}} d λ (x)$
	$= β A_{n} k^{- θ} \int \frac{𝟏_{(k^{- γ}, \infty)} r^{n - 1}}{{(k^{- 1} + r)}^{n + θ}} d λ (r)$
	$\leq β A_{n} k^{- θ} \int 𝟏_{(k^{- γ}, \infty)} r^{- θ - 1} d λ (r)$
	$= β A_{n} k^{θ (γ - 1)} / θ ,$

where $A_{n}$ is the surface area of a unit sphere embedded in $ℝ^{n}$ . We then have

	${‖ g_{k} ⋆ (𝟏_{𝕂} f) ‖}_{ℒ_{1} (𝕂_{k}^{∁})}$
	$= \int \int 𝟏_{{y \in 𝕂}} 𝟏_{{x \in 𝕂_{k}^{∁}}} f (y) g_{k} (x - y) d λ (x) d λ (y)$
	$\leq {‖ 𝟏_{𝕂} f ‖}_{ℒ_{\infty}} \int \int 𝟏_{{y \in 𝕂}} \frac{𝟏_{{x : {‖ x - y ‖}_{2} > k^{- γ}}} β k^{- θ}}{{(k^{- 1} + {‖ x - y ‖}_{2})}^{n + θ}} d λ (x) d λ (y)$
	$\leq {‖ 𝟏_{𝕂} f ‖}_{ℒ_{\infty}} λ (𝕂) β A_{n} k^{θ (γ - 1)} / θ ,$

which implies that we can choose a $k_{3} \in ℕ$ , such that for all $k \geq k_{3}$ ,

{‖ g_{k} ⋆ (𝟏_{𝕂} ​ f) ‖}_{ℒ_{1} ​ (𝕂_{k}^{∁})} < \frac{ϵ}{24} ​ .

(22)

Lastly, we write

	${‖ h_{m - 1}^{g} ‖}_{ℒ_{1} (𝕂_{k}^{∁})}$
	$= \int 𝟏_{𝕂_{k}^{∁}} \sum_{i = 1}^{m - 1} c_{i} k^{n} g (k x - z_{i}) d λ$
	$= \sum_{i = 1}^{m - 1} \int 𝟏_{𝕂_{k}^{∁}} [k^{- n} \int 𝟏_{𝔸_{i}} f (\frac{z}{k}) d λ (z)] k^{n} g (k x - z_{i}) d λ (x)$
	$\leq {‖ 𝟏_{𝕂} f ‖}_{ℒ_{\infty}} \sum_{i = 1}^{m - 1} k^{- n} λ (𝔸_{i}) \int 𝟏_{𝕂_{k}^{∁}} g_{k} (x - \frac{z_{i}}{k}) d λ$
	$\leq {‖ 𝟏_{𝕂} f ‖}_{ℒ_{\infty}} \sum_{i = 1}^{m - 1} k^{- n} λ (𝔸_{i}) \frac{β A_{n} k^{θ (γ - 1)}}{θ}$
	$\leq {‖ 𝟏_{𝕂} f ‖}_{ℒ_{\infty}} λ (𝕂) β A_{n} k^{θ (γ - 1)} / θ ,$

which implies that we can choose the same $k_{3}$ as above to obtain the bound

{‖ h_{m - 1}^{g} ‖}_{ℒ_{1} ​ (𝕂_{k}^{∁})} < \frac{ϵ}{24} ​,

(23)

for any $k \geq k_{3}$ .

Thus, we obtain the bound ${‖ 𝟏_{𝕂} f - h_{m}^{g} ‖}_{ℒ_{1}} < ϵ / 2$ , for all $k \geq \max {k_{1}, k_{2}, k_{3}}$ , by combining (18), (19), (20), (21), (22), and (23), via the triangle inequality. The result is proved by combing the bound above, with (16), for an appropriately large $r (ϵ) \in ℕ$ .

3 Comments and discussion

3.1 Relationship to Theorem 1

In the proof of Theorem 1, the famous Hilbert space approximation result of [16] and [2] was used to bound the $ℒ_{2}$ norm between any approximand $f \in ℒ_{2}$ and a convex combination of bounded functions in $ℒ_{2}$ . This approximation theorem is exactly the $p = 2$ case of the more general theorem of [9], as presented in Lemma 11. Thus, one can view Theorem 5(c) as the $p \in (1, \infty)$ generalization of Theorem 1.

3.2 The class $𝒲$ is a proper subset of the class $𝒞_{0}$

Here, we comment on the nature of class $𝒲$ , which was investigated by [1] and [26]. We recall that [1] conjectured that Theorem 4 generalizes from $g = ϕ$ to $g \in 𝒱$ . In Theorem 5(a)–(e), we assume that $g \in 𝒞_{0}$ . We can demonstrate that $g \in 𝒞_{0}$ is a strictly weaker condition than $g \in 𝒱$ or $g \in 𝒲$ .

For example, consider the function in $g : ℝ \to ℝ$ such that $g (x) = 0$ if $x < 0$ and

g ​ (x)

= \sum_{i = 1}^{\infty} \frac{2^{2 ​ i}}{i} ​ [{(x - i + 1)}^{2 ​ i} ​ 𝟏_{{i - 1 \leq x < i - 1 / 2}} + {(x - i)}^{2 ​ i} ​ 𝟏_{{i - 1 / 2 \leq x < i}}] ​ if ​ x \geq 0 ​,

and note that

\int 𝟏_{(- 1 / 2, 1 / 2)} ​ \frac{{(2 ​ x)}^{2 ​ i}}{i} ​ d ​ λ = \frac{1}{2 ​ i^{2} + i} < \frac{1}{i^{2}} ​ .

Since $\sum_{i = 1}^{\infty} (1 / i^{2}) = π^{2} / 6$ , $g \in ℒ_{1}$ . Furthermore, $g$ is continuous since all stationary points of $g$ are continuous. In $ℝ$ , $g \in 𝒞_{0}$ if

lim_{x \to \pm \infty} g ​ (x) = 0 ​ .

For $x \leq 0$ , we observe that $g = 0$ and thus the left limit is satisfied. On the right, for any $1 / ϵ > 0$ , we have $x (ϵ) \geq ⌈ ϵ ⌉ - 1 / 2$ , so that $g (x) < 1 / ϵ$ , for all $x > x (ϵ)$ , where $⌈ \cdot ⌉$ is the ceiling operator. Therefore, $g \in 𝒞_{0}$ .

Within each interval $i - 1 \leq x < i$ , we observe that $g$ is locally maximized at $x = i - 1 / 2$ . The local maximum corresponding to each of these points is $1 / i$ . Thus $g \notin 𝒲$ , since

\sum_{i = 1}^{\infty} \frac{1}{i} ​ < \sum_{y \in ℤ} sup_{x \in [0, 1]} | ​ g ​ (x + y) |,

where $\sum_{i = 1}^{\infty} (1 / i) = \infty$ . Furthermore, $g \notin 𝒱$ since $𝒱 \subset 𝒲$ .

3.3 Convergence in measure

Along with the conclusions of Theorem 5(d) and (e), Lemma 12 also implies convergence in measure. That is, if $ν$ is a $σ -finite$ Borel measure on $ℝ^{n}$ , then for any $ν -measurable$ $f$ , there exists a sequence ${h_{m}^{g}}$ , such that for any $ϵ > 0$ ,

lim_{m \to \infty} υ ​ ({x \in ℝ^{n} : | f ​ (x) - h_{m}^{g} ​ (x) | \geq ϵ}) = 0 ​ .

Appendix A Technical results

Throughout the main text, we utilize a number of established technical results. For the convenience of the reader, we append these results within this Appendix. Sources from which we draw the unproved results are provided at the end of the section.

Lemma 5.

Let ${g_{k}}$ be a sequence of pdfs in $ℒ_{1}$ and for every $δ > 0$

lim_{k \to \infty} \int 𝟏_{{x : {‖ x ‖}_{2} > δ}} ​ g_{k} ​ d ​ λ = 0 ​ .

Then, for all $f \in ℒ_{p}$ and $1 \leq p < \infty$ ,

lim_{k \to \infty} {‖ g_{k} ⋆ f - f ‖}_{ℒ_{p}} = 0 ​ .

Furthermore, for all $f \in 𝒞_{b}$ and any compact $𝕂 \subset ℝ^{n}$ ,

lim_{k \to \infty} {‖ g_{k} ⋆ f - f ‖}_{ℒ_{\infty} ​ (𝕂)} = 0 ​ .

The sequences ${g_{k}}$ from Lemma 5 are often called approximate identities or approximations of the identity. A simple construction of approximate identities is by taking dilations $g_{k} (x) = k^{n} g (k x)$ , which yields the following corollary.

Corollary 1.

Let $g$ be a pdf. Then the sequence of dilations ${g_{k} : g_{k} (x) = k^{n} g (k x)}$ , satisfies the hypothesis of Lemma 5 and hence permits its conclusion.

Lemma 6.

The class $𝒞_{0}$ is a subset of $𝒞_{b}$ . Furthermore, if $f \in 𝒞_{0}$ , then $f$ is uniformly continuous.

Lemma 7 (Urysohn’s Lemma).

If $𝕂 \subset ℝ^{n}$ is compact, then there exists some $g \in 𝒞_{c}$ , such that $0 \leq g \leq 1$ and $𝟏_{𝕂} g = 1$ .

Lemma 8.

If $𝕏 \subset ℝ^{n}$ is bounded, then for any $r > 0$ , $𝕏$ can be covered by $⋃_{i = 1}^{m} 𝔹 (x_{i}, r)$ for some finite $m \in ℕ$ , where $x_{i} \in ℝ^{n}$ and $i \in [m]$ .

Lemma 9.

If $0 < p < q < r \leq \infty$ , then $ℒ_{p} \cap ℒ_{r} \subset ℒ_{q}$ .

Let $Γ : ℝ \to ℝ$ be the usual gamma function, defined as $Γ (z) = \int 𝟏_{(0, \infty)} x^{z - 1} \exp (- x) d λ$ .

Lemma 10.

If $f \in ℒ_{p}$ and $g \in ℒ_{1}$ , for $1 \leq p \leq \infty$ , then $f ⋆ g$ exists and we have ${‖ f ⋆ g ‖}_{ℒ_{p}} \leq {‖ g ‖}_{ℒ_{1}} {‖ f ‖}_{ℒ_{p}}$ .

Lemma 11.

Let $𝒢 \subset ℒ_{p}$ , for some $1 \leq p < \infty$ , and let $f \in \bar{Conv} (𝒢)$ . For any $K > 0$ , such that ${‖ f - α ‖}_{ℒ_{p}} < K$ , for all $α \in 𝒢$ , there exists a $h_{m} \in {Conv}_{m} (𝒢)$ , such that

{‖ f - h_{m} ‖}_{ℒ_{p}} \leq \frac{C_{p} ​ K}{m^{1 - 1 / α}} ​,

where $α = \min {p, 2}$ , and

C_{p} = {\begin{matrix} 1 & if ​ 1 \leq p \leq 2 ​, \\ \sqrt{2} ​ {[\sqrt{π} ​ Γ ​ (\frac{p + 1}{2})]}^{1 / p} & if ​ p > 2 ​ . \end{matrix}

Lemma 12.

In any measure $ν$ , uniform convergence implies almost uniform convergence, and almost uniform convergence implies almost everywhere convergence and convergence in measure, with respect to $ν$ .

Appendix B Sources of results

Lemma 5 is reported as Theorem 9.3.3 in [21] (see also Theorem 2 of [4, Ch. 20]). The proof of Corollary 1 can be taken from that of Theorem 4 of [4, Ch. 20]. Lemma 6 appears in [5], as Proposition 1.4.5. Lemma 7 is taken from Corollary 1.2.9 of [5]. Lemma 8 appears as Theorem 1.2.2 in [5]. Lemma 9 can be found in [12, Prop. 6.10]. Lemma 10 can be found in [21, Thm. 9.3.1]. Lemma 11 appears as Corollary 2.6 in [9]. Lemma 12 can be obtained from the definition of almost uniform convergence, Lemma 7.10, and Theorem 7.11 of [3].

Acknowledgment

HDN is personally funded by Australian Research Council (ARC) grant DE170101134. HDN and GJM are supported by ARC grant DP180101192. FC is supported by Agence Nationale de la Recherche (ANR) grant SMILES ANR-18-CE40-0014 and by Région Normandie grant RIN AStERiCs.

References

[1] A G Bacharoglou. Approximation of probability distributions by convex mixtures of Gaussian measures. Proceedings of the American Mathematical Society, 138:2619–2628, 2010. ↩ 1 3.2
[2] A R Barron. Universal approximation bound for superpositions of a sigmoidal function. IEEE Transactions on Information Theory, 39:930–945, 1993. ↩ 2.4 3.1
[3] R G Bartle. The Elements of Integration and Lesbegue Measure. Wiley, New York, 1995. ↩ B
[4] W Cheney and W Light. A Course in Approximation Theory. Brooks/Cole, Pacific Grove, 2000. ↩ 1 2.1 B
[5] J B Conway. A Course in Abstract Analysis. American Mathematical Society, Providence, 2012. ↩ B
[6] A DasGupta. Asymptotic Theory Of Statistics And Probability. Springer, New York, 2008. ↩ 1
[7] L De Branges. The Stone-Weierstrass theorem. Proceedings of the American Mathematical Society, 10:822–824, 1959. ↩ 1
[8] L Devroye and G Lugosi. Combinatorial Methods in Density Estimation. Springer, New York, 2000. ↩ 1
[9] M J Donahue, L Gurvits, C Darken, and E Sontag. Rates of convex approximation in non-Hilbert spaces. Constructive Approximation, 13:187–220, 1997. ↩ 1 2.4 3.1 B
[10] B S Everitt and D J Hand. Finite Mixture Distributions. Chapman and Hall, London, 1981. ↩ 1
[11] H G Feichtinger. A characterization of Wiener’s algebra on locally compact groups. Archiv der Mathematik, 29:136–140, 1977. ↩ 1
[12] G B Folland. Real Analysis: Modern Techniques and Their Applications. Wiley, New York, 1999. ↩ B
[13] F Forbes and D Wraith. A new family of multivariate heavy-tailed distributions with variable marginal amounts of tailweights. Statistics and Computing, 24:971–984, 2013. ↩ 1
[14] S Fruwirth-Schnatter. Finite Mixture and Markov Switching Models. Springer, New York, 2006. ↩ 1
[15] S Fruwirth-Schnatter, G Celeux, and C P Robert, editors. Handbook of Mixture Analysis. CRC Press, Boca Raton, 2019. ↩ 1
[16] L K Jones. A simple lemma on greedy approximation in Hilbert space and convergene rates for projection pursuit regression and neural network training. Annals of Statistics, 20:608–613, 1992. ↩ 2.4 3.1
[17] S Kullback and R A Leibler. On information and sufficiency. Annals of Mathematical Statistics, 22:79–86, 1951. ↩ 1
[18] S X Lee and G J McLachlan. Finite mixtures of canonical fundamental skew t-distributions: the unification of the trstricted and unrestricted skew t-mixture models. Statistics and Computing, 26:573–589, 2016. ↩ 1
[19] J Q Li and A R Barron. Mixture density estimation. In S A Solla, T K Leen, and K R Mueller, editors, Advances in Neural Information Processing Systems, volume 12, Cambridge, 1999. MIT Press. ↩ 1
[20] B G Lindsay. Mixture models: theory, geometry and applications. In NSF-CBMS Regional Conference Series in Probability and Statistics, 1995. ↩ 1
[21] B Makarov and A Podkorytov. Real Analysis: Measures, Integrals and Applications. Springer, London, 2013. ↩ 2.1 B
[22] G J McLachlan and K E Basford. Mixture Models: Inference and Applications to Clustering. Marcel Dekker, New York, 1988. ↩ 1
[23] G J McLachlan and D Peel. Finite Mixture Models. Wiley, New York, 2000. ↩ 1
[24] Kerrie L Mengersen, Christian Robert, and Mike Titterington. Mixtures: Estimation and Applications. Wiley, New York, 2011. ↩ 1
[25] V Nestoridis and C Papadimitropoulos. Abstract theory of universal series and an application to Dirichlet series. Comptes Rendus Academy of Science Paris Series I, 341:530–543, 2005. ↩ 1
[26] V Nestoridis, S Schmutzhard, and V Stefanopoulos. Universal series induced by approximate identities and some relevant applications. Journal of Approximation Theory, 163:1783–1797, 2011. ↩ 1 2.1 3.2
[27] V Nestoridis and V Stefanopoulos. Universial series and approximate identities. Technical report, Department of Mathematics and Statistics, University of Cyprus, 2007. ↩ 1 2.1
[28] H D Nguyen and G J McLachlan. On approximations via convolution-defined mixture models. Communications in Statistics - Theory and Methods, 2019. In press. ↩ 1
[29] D Peel and G J McLachlan. Robust mixture modelling using the t distribution. Statistics and Computing, 10:339–348, 2000. ↩ 1
[30] A Rakhlin, D Panchenko, and S Mukherjee. Risk bounds for mixture density estimation. ESAIM: Probability and Statistics, 9:220–229, 2005. ↩ 1
[31] W Rudin. Principles of Mathematical Analysis. McGraw-Hill, Singapore, 1976. ↩ 2.1
[32] I W Sandberg. Gaussian radial basis functions and inner product sspace. Circuits, Systems and Signal Processing, 20:635–642, 2001. ↩ 1
[33] P Schlattmann. Medical Applications of Finite Mixture Models. Springer, Berlin, 2009. ↩ 1
[34] M H Stone. The generalized Weierstrass approximation theorem. Mathematical Magazine, 21:237–254, 1948. ↩ 1
[35] D M Titterington, A F M Smith, and U E Makov. Statistical Analysis of Finite Mixture Distributions. Wiley, New York, 1985. ↩ 1
[36] S van de Geer. Asymptotic theory for maximum likelihood in nonparametric mixture models. Computational Statistics and Data Analysis, 41:453–464, 2003. ↩ 2.4
[37] J L Walker and M Ben-Akiva. Advances in discrete choice: mixture models. In Andre De Palma, Robin Lindsey, Emile Quinet, and Roger Vickerman, editors, A Handbook of Transport Economics, pages 160–187. Edward Edgar, Cheltenham, 2011. ↩ 1
[38] Y Xu, W A Light, and E W Cheney. Constructive methods of approximation by ridge functions and radial functions. Numerical Algorithms, 4:205–223, 1993. ↩ 1
[39] G Yona. Introduction to Computational Proteomics. CRC Press, Boca Raton, 2011. ↩ 1
[40] A J Zeevi and R Meir. Density estimation through convex combinations of densities: approximation and estimation bounds. Neural Computation, 10:99–109, 1997. ↩ 1

[bib.bib1] [1] A G Bacharoglou. Approximation of probability distributions by convex mixtures of Gaussian measures. Proceedings of the American Mathematical Society, 138:2619–2628, 2010. ↩ 1 3.2

[bib.bib2] [2] A R Barron. Universal approximation bound for superpositions of a sigmoidal function. IEEE Transactions on Information Theory, 39:930–945, 1993. ↩ 2.4 3.1

[bib.bib3] [3] R G Bartle. The Elements of Integration and Lesbegue Measure. Wiley, New York, 1995. ↩ B

[bib.bib4] [4] W Cheney and W Light. A Course in Approximation Theory. Brooks/Cole, Pacific Grove, 2000. ↩ 1 2.1 B

[bib.bib5] [5] J B Conway. A Course in Abstract Analysis. American Mathematical Society, Providence, 2012. ↩ B

[bib.bib6] [6] A DasGupta. Asymptotic Theory Of Statistics And Probability. Springer, New York, 2008. ↩ 1

[bib.bib7] [7] L De Branges. The Stone-Weierstrass theorem. Proceedings of the American Mathematical Society, 10:822–824, 1959. ↩ 1

[bib.bib8] [8] L Devroye and G Lugosi. Combinatorial Methods in Density Estimation. Springer, New York, 2000. ↩ 1

[bib.bib9] [9] M J Donahue, L Gurvits, C Darken, and E Sontag. Rates of convex approximation in non-Hilbert spaces. Constructive Approximation, 13:187–220, 1997. ↩ 1 2.4 3.1 B

[bib.bib10] [10] B S Everitt and D J Hand. Finite Mixture Distributions. Chapman and Hall, London, 1981. ↩ 1

[bib.bib11] [11] H G Feichtinger. A characterization of Wiener’s algebra on locally compact groups. Archiv der Mathematik, 29:136–140, 1977. ↩ 1

[bib.bib12] [12] G B Folland. Real Analysis: Modern Techniques and Their Applications. Wiley, New York, 1999. ↩ B

[bib.bib13] [13] F Forbes and D Wraith. A new family of multivariate heavy-tailed distributions with variable marginal amounts of tailweights. Statistics and Computing, 24:971–984, 2013. ↩ 1

[bib.bib14] [14] S Fruwirth-Schnatter. Finite Mixture and Markov Switching Models. Springer, New York, 2006. ↩ 1

[bib.bib15] [15] S Fruwirth-Schnatter, G Celeux, and C P Robert, editors. Handbook of Mixture Analysis. CRC Press, Boca Raton, 2019. ↩ 1

[bib.bib16] [16] L K Jones. A simple lemma on greedy approximation in Hilbert space and convergene rates for projection pursuit regression and neural network training. Annals of Statistics, 20:608–613, 1992. ↩ 2.4 3.1

[bib.bib17] [17] S Kullback and R A Leibler. On information and sufficiency. Annals of Mathematical Statistics, 22:79–86, 1951. ↩ 1

[bib.bib18] [18] S X Lee and G J McLachlan. Finite mixtures of canonical fundamental skew t-distributions: the unification of the trstricted and unrestricted skew t-mixture models. Statistics and Computing, 26:573–589, 2016. ↩ 1

[bib.bib19] [19] J Q Li and A R Barron. Mixture density estimation. In S A Solla, T K Leen, and K R Mueller, editors, Advances in Neural Information Processing Systems, volume 12, Cambridge, 1999. MIT Press. ↩ 1

[bib.bib20] [20] B G Lindsay. Mixture models: theory, geometry and applications. In NSF-CBMS Regional Conference Series in Probability and Statistics, 1995. ↩ 1

[bib.bib21] [21] B Makarov and A Podkorytov. Real Analysis: Measures, Integrals and Applications. Springer, London, 2013. ↩ 2.1 B

[bib.bib22] [22] G J McLachlan and K E Basford. Mixture Models: Inference and Applications to Clustering. Marcel Dekker, New York, 1988. ↩ 1

[bib.bib23] [23] G J McLachlan and D Peel. Finite Mixture Models. Wiley, New York, 2000. ↩ 1

[bib.bib24] [24] Kerrie L Mengersen, Christian Robert, and Mike Titterington. Mixtures: Estimation and Applications. Wiley, New York, 2011. ↩ 1

[bib.bib25] [25] V Nestoridis and C Papadimitropoulos. Abstract theory of universal series and an application to Dirichlet series. Comptes Rendus Academy of Science Paris Series I, 341:530–543, 2005. ↩ 1

[bib.bib26] [26] V Nestoridis, S Schmutzhard, and V Stefanopoulos. Universal series induced by approximate identities and some relevant applications. Journal of Approximation Theory, 163:1783–1797, 2011. ↩ 1 2.1 3.2

[bib.bib27] [27] V Nestoridis and V Stefanopoulos. Universial series and approximate identities. Technical report, Department of Mathematics and Statistics, University of Cyprus, 2007. ↩ 1 2.1

[bib.bib28] [28] H D Nguyen and G J McLachlan. On approximations via convolution-defined mixture models. Communications in Statistics - Theory and Methods, 2019. In press. ↩ 1

[bib.bib29] [29] D Peel and G J McLachlan. Robust mixture modelling using the t distribution. Statistics and Computing, 10:339–348, 2000. ↩ 1

[bib.bib30] [30] A Rakhlin, D Panchenko, and S Mukherjee. Risk bounds for mixture density estimation. ESAIM: Probability and Statistics, 9:220–229, 2005. ↩ 1

[bib.bib31] [31] W Rudin. Principles of Mathematical Analysis. McGraw-Hill, Singapore, 1976. ↩ 2.1

[bib.bib32] [32] I W Sandberg. Gaussian radial basis functions and inner product sspace. Circuits, Systems and Signal Processing, 20:635–642, 2001. ↩ 1

[bib.bib33] [33] P Schlattmann. Medical Applications of Finite Mixture Models. Springer, Berlin, 2009. ↩ 1

[bib.bib34] [34] M H Stone. The generalized Weierstrass approximation theorem. Mathematical Magazine, 21:237–254, 1948. ↩ 1

[bib.bib35] [35] D M Titterington, A F M Smith, and U E Makov. Statistical Analysis of Finite Mixture Distributions. Wiley, New York, 1985. ↩ 1

[bib.bib36] [36] S van de Geer. Asymptotic theory for maximum likelihood in nonparametric mixture models. Computational Statistics and Data Analysis, 41:453–464, 2003. ↩ 2.4

[bib.bib37] [37] J L Walker and M Ben-Akiva. Advances in discrete choice: mixture models. In Andre De Palma, Robin Lindsey, Emile Quinet, and Roger Vickerman, editors, A Handbook of Transport Economics, pages 160–187. Edward Edgar, Cheltenham, 2011. ↩ 1

[bib.bib38] [38] Y Xu, W A Light, and E W Cheney. Constructive methods of approximation by ridge functions and radial functions. Numerical Algorithms, 4:205–223, 1993. ↩ 1

[bib.bib39] [39] G Yona. Introduction to Computational Proteomics. CRC Press, Boca Raton, 2011. ↩ 1

[bib.bib40] [40] A J Zeevi and R Meir. Density estimation through convex combinations of densities: approximation and estimation bounds. Neural Computation, 10:99–109, 1997. ↩ 1

Approximation by Finite Mixtures of Continuous Density Functions That Vanish at Infinity

Abstract

1 Introduction

Theorem 1 (Zeevi and Meir, 1997).

Claim 1.

Theorem 2 (Nestoridis and Stefanopoulos, 2007, Thm. 3.2).

Theorem 3 (Nestoridis et al., 2011, Thm. 3.2).

Theorem 4 (Bacharoglou, 2010, Cor. 2.5).

2 Main result

Theorem 5 (Main result).

2.1 Technical preliminaries

Lemma 1.

Proof.

Lemma 2.

Proof.

Lemma 3.

Proof.

Lemma 4.

Proof.

2.2 Proof of Theorem 5(a)

2.3 Proof of Theorem 5(b)

2.4 Proof of Theorem 5(c)

2.5 Proof of Theorem 5(d) and Theorem 5(e)

2.6 Proof of Theorem 5(f)

3 Comments and discussion

3.1 Relationship to Theorem 1

3.2 The class 𝒲 is a proper subset of the class 𝒞0

3.3 Convergence in measure

Appendix A Technical results

Lemma 5.

Corollary 1.

Lemma 6.

Lemma 7 (Urysohn’s Lemma).

Lemma 8.

Lemma 9.

Lemma 10.

Lemma 11.

Lemma 12.

Appendix B Sources of results

Acknowledgment

References

Cite this paper

Related work

3.2 The class $𝒲$ is a proper subset of the class $𝒞_{0}$