GEODETICALLY CONVEX SETS IN THE HEISENBERG GROUP

ROBERTO MONTI AND MATTHIEU RICKLY

In memory of Matthias Studer

Abstract. We prove that the geodetic envelope of a subset of the Heisenberg group containing three points not lying on the same geodesic is the whole group. As a corollary, we obtain that a function on the group which is convex along geodesics must be constant.

1. Introduction

Recently, several notions of convexity have been introduced and studied in Heisenberg groups and in more general Carnot groups. A weak and a strong definition of convex function are discussed in [DGN]: roughly speaking, the function is required to be convex along the integral curves of the left invariant horizontal vector fields. A different approach has been proposed in [LMS], where the notion of convexity in the viscosity sense has been transposed from the Euclidean to the sub–Riemannian setting. Relations between convexity in the viscosity sense and horizontal convexity as well as sharp regularity properties of convex functions in Heisenberg groups are discussed in [LMS] and [BR].

The main motivation for the study of convexity in sub–Riemannian geometries is the development of a theory of fully non–linear sub–elliptic PDE’s. For instance, it has been recently proved in [GM] that convex functions in the Heisenberg group naturally define a Monge–Ampère measure satisfying a comparison principle (see also [GT]).

In this paper we discuss a definition of convex set of geometric type. The Heisenberg group ℍ ≅ ℝ^{3} can be endowed with a left invariant metric d which is known as Carnot–Carathéodory metric. The resulting metric space (ℍ, d) is geodesic: every pair of points can be connected by at least one geodesic. As in Euclidean space, a subset A ⊂ ℍ can be defined to be geodetically convex if the image of any geodesic connecting two elements of A is contained in A. We describe the family of geodetically convex sets showing that, differently from the Euclidean case, it is very poor. Indeed, the geodetic convex envelope of a set consisting of merely two points can be the whole group:

Theorem 1.1. Let A = {(x, y, t_{1}), (x, y, t_{2})} ⊆ ℍ ≅ ℝ^{3} where t_{1} ≠ t_{2}. Then the smallest geodetically convex set containing A is ℍ.

This is due to the fact that pairs of points with the same projection on the (x, y)– plane admit an infinite number of geodesics connecting them. The proof of Theorem 1.1 is given in §3.1. A consequence of Theorem 1.1 is the following

Theorem 1.2. Let p_{1}, p_{2}, p_{3} ∈ ℍ be three points not lying on the same geodesic. Then the smallest geodetically convex set containing {p_{1}, p_{2}, p_{3}} is ℍ.

Thus, the only geodetically convex subsets of ℍ are the empty set, points, arcs of geodesics and ℍ. The proof of Theorem 1.2 is given in §3.2.

The lack of geodetically convex sets has its counterpart in the lack of geodetically convex functions on ℍ. A function u : ℍ → ℝ is said to be geodetically convex if for any p_{0}, p_{1} ∈ ℍ and any geodesic γ : [0, d(p_{0}, p_{1})] → ℍ ≅ ℝ^{3} parameterized by arc–length connecting p_{0} and p_{1}, the function t ↦ u(γ(t)) is convex in the usual sense.

Corollary 1.3. If u : ℍ → ℝ is geodetically convex, then u is constant.

The proof is given in §3.3.

2. Basic facts concerning geodesics

We recall the definition and the properties of geodesics in the Heisenberg group needed in the sequel. In the following,

ℍ ≅ ℝ^{3} = {(x, y, t) | x, y, t ∈ ℝ}

denotes the Heisenberg group with the group law

(x, y, t) ∗ (x′, y′, t′) = (x + x′, y + y′, t + t′ + 2(x′y − xy′)) .

One can check that the unit element is 0 ∈ ℝ^{3} and that the inverse of p = (x, y, t) is p^{−1} = (−x, −y, −t). The t–axis Z = {(0, 0, t) | t ∈ ℝ} is the center of the group. In the setup of the Heisenberg group, Euclidean translations and dilations are replaced by the left translations l_{p} : ℍ → ℍ, p ∈ ℍ, defined by

l_{p}(q) = p ∗ q,

and by the anisotropic dilations δ_{r} : ℍ → ℍ, r > 0, defined by

δ_{r}(x, y, t) = (rx, ry, r^{2}t).

Clearly, (δ_{r})_{r>0} is a group of automorphisms of ℍ.

The differential structure on ℍ is determined by the left invariant vector fields

X = \frac{∂}{∂x} + 2y \frac{∂}{∂t}, Y = \frac{∂}{∂y} − 2x\frac{∂}{∂t} and T = \frac{∂}{∂t},

where X and Y are the so–called horizontal vector fields. The pointwise linear span of X and Y forms a 2–dimensional bundle on ℝ^{3} called horizontal bundle or horizontal distribution. The only non–trivial bracket relation is [X, Y] = −4T. Thus the distribution is bracket–generating.

A Lipschitz curve γ : [0, 1] → ℍ is said to be horizontal if

γ̇(t) ∈ span_{ℝ}{X(γ(t)), Y (γ(t))}, γ̇(t) = h_{1}(t)X(γ(t)) + h_{2}(t)Y (γ(t)),

for almost every t ∈ [0, 1]. The length of a horizontal curve γ : [0, 1] → ℍ is defined to be

L(γ) = ∫_{0}^{1}(h_{1}^{2}(t) + h_{2}^{2}(t))^{\frac{1}{2}} dt.

Given p_{1}, p_{2} ∈ ℍ, define

d(p_{1}, p_{2}) := inf{L(γ) | γ is a horizontal curve connecting p_{1} and p_{2}}.

It is well–known that d is a metric on ℍ, the so–called Carnot–Carathéodory distance, which is left–invariant, i.e. d(l_{p}(p_{1}), l_{p}(p_{2})) = d(p_{1}, p_{2}) for all p, p_{1}, p_{2} ∈ ℍ, homogeneous, i.e. d(δ_{r}(p_{1}), δ_{r}(p_{2})) = rd(p_{1}, p_{2}) for all p_{1}, p_{2} ∈ ℍ and r > 0, and which induces the Euclidean topology on ℝ^{3}.

Definition 2.1. A geodesic connecting two points p_{0}, p_{1} ∈ ℍ is a length minimizing horizontal curve γ : [0, T] → ℍ, i.e. a curve such that γ(0) = p_{0}, γ(T) = p_{1} and L(γ) = d(p_{0}, p_{1}).

It can be shown that (ℍ, d) is complete and that any two points can be connected by a (not necessarily unique) geodesic. Geodesics can be computed explicitly and we refer, for instance, to [G], [K], [B], [M] or [AR] for a discussion of the problem. Precisely, geodesics starting from the origin 0 ∈ ℍ are smooth curves γ = (γ_{1}, γ_{1}, γ_{3}) of the form

γ_{1}(t) = \frac{α sin(φt) + β(1 − cos(φt))}{φ}

γ_{2}(t) = \frac{β sin(φt) − α(1 − cos(φt))}{φ}

γ_{3}(t) = 2\frac{φt − sin(φt)}{φ^{2}}.

(2.1)

The real parameters α, β, φ specify the geodesic. It is useful to work with geodesics parameterized by arc–length. The condition ensuring arc–length parametrization is α^{2} + β^{2} = 1 and the curve must be consequently defined on an interval [0, L] where L = L(γ). With this choice t ≥ 0 is the arc–length parameter. In the case φ = 0, the formulae (2.1) must be understood in the limit sense and the curve γ is a geodesic on [0, L] for any 0 ≤ L < +∞. If φ ≠ 0 the curve γ in (2.1) is length minimizing on [0, L] if and only if L ≤ 2π/|φ|. For L > 2π/|φ| the curve γ : [0, L] → ℍ is not a geodesic anymore.

Geodesics starting from an arbitrary point can be recovered from (2.1) by left translations. Note that isometries of (ℍ, d) and dilations transform geodesics into geodesics.

In the following proposition, we list some known properties of geodesics that can be derived from (2.1) and will be used in the sequel.

Proposition 2.1.

(i) For any p ∈ ℍ \ Z there exists a unique geodesic connecting 0 and p. (ii) For any p ∈ Z, p ≠ 0, and for any pair (α, β) ∈ ℝ^{2} with α^{2}+β^{2} = 1, there exists

a unique geodesic γ connecting 0 and p such that γ̇(0) = α∂_{x} + β∂_{y} = (α, β, 0). Moreover, the union of all the images of geodesics connecting 0 and p is the boundary of a convex open set which is invariant with respect to rotations of ℝ^{3} that fix Z.

(iii) The image of the geodesic connecting the point p = (x, y, t) with the point

p^{∗} = (−x, −y, t) is the line segment [p^{∗}, p]. (iv) For φ ≠ 0 the projection onto the (x, y)–plane of the geodesic γ in (2.1) is an

arc of circle with radius 1/|φ|.

(v) A geodesic γ : [0, L] → ℍ with parameter φ ∈ ℝ \ {0} and L < 2π/|φ| –respectively φ = 0 and L ≥ 0– can be uniquely extended on [0, 2π/|φ|] – respectively on [0, L̃] for any L̃ ≥ L.

(vi) The mapping Φ : \{(α, β, φ, t) | α^{2} + β^{2} = 1, φ ∈ ℝ, t ∈ (0, \frac{2π}{|φ|})\} → ℍ \ Z given by

Φ(α, β, φ, t) = (\frac{α sin φt + β(1 − cos φt)}{φ}, \frac{β sin φt − α(1 − cos φt)}{φ}, 2\frac{φt − sin φt}{φ^{2}})

is a homeomorphism.

Let us now state some definitions and preliminary results that will be needed in the proofs of Theorem 1.1 and 1.2.

Definition 2.2. We say that a set C ⊆ ℍ is geodetically convex if for all p_{0}, p_{1} ∈ C and all geodesics γ : [0, L] → ℍ with L = d(p_{0}, p_{1}), γ(0) = p_{0} and γ(L) = p_{1} we have γ([0, L]) ⊆ C. The geodetic convex envelope C(A) of A ⊆ ℍ is the smallest geodetically convex subset of ℍ containing A.

Definition 2.3. For p_{0}, p_{1} ∈ ℍ, G(p_{0}, p_{1}) denotes the set of images of geodesics connecting p_{0} and p_{1}. Given A ⊆ ℍ we define G(A) := ⋃_{p_{0},p_{1}∈A} G(p_{0}, p_{1}), G^{0}(A) := A and G^{n+1}(A) := G(G^{n}(A)) for all n ∈ ℕ_{0}.

Lemma 2.2. For A ⊆ ℍ we have C(A) = ⋃_{n∈ℕ_{0}} G^{n}(A).

Proof. Using the fact that G^{n}(A) ⊆ G^{n+1}(A) for all n ∈ ℕ_{0}, one easily checks that ⋃_{n∈ℕ_{0}} G^{n}(A) is geodetically convex and contains A. This gives C(A) ⊆ ⋃_{n∈ℕ_{0}} G^{n}(A). On the other hand A ⊆ C(A), and if G^{n}(A) ⊆ C(A) for some n ∈ ℕ_{0}, then G^{n+1}(A) ⊆ C(A).

In the following R denotes the set of rotations of ℝ^{3} that fix the center Z. Precisely,

(2.2) R = \{R_{ϑ} = ( \begin{matrix}cos(ϑ) & sin(ϑ) & 0 \\ − sin(ϑ) & cos(ϑ) & 0 \\ 0 & 0 & 1 \end{matrix} ) | ϑ ∈ [0, 2π)\} .

The rotations R_{ϑ} are isometries of the Heisenberg group endowed with the Carnot– Carathéodory metric. The map G : ℍ → ℍ given by

G(x, y, t) = (−x, y, −t)

is also an isometry.

Lemma 2.3. Let A ⊆ ℍ, p ∈ ℍ, r > 0 and R ∈ R. Then C(l_{p}(A)) = l_{p}(C(A)), C(δ_{r}(A)) = δ_{r}(C(A)), C(R(A)) = R(C(A)) and C(G(A)) = G(C(A)).

Proof. We prove the statement for R. First of all note that isometries of (ℍ, d) and dilations map geodesics to geodesics. It is easy to prove by induction that

R(G^{n}(A)) = G^{n}(R(A)), n ∈ ℕ_{0}.

Moreover, by Lemma 2.2

R(C(A)) = R ( ⋃_{n∈ℕ_{0}} G^{n}(A)) = ⋃_{n∈ℕ_{0}} R(G^{n}(A)) = ⋃_{n∈ℕ_{0}} G^{n}(R(A)) = C(R(A)).

Lemma 2.4. We have {(0, 0, t) | − 1 ≤ t ≤ 1} ⊆ G^{2}({(0, 0, −1), (0, 0, 1)}).

Proof. By Proposition 2.1 (ii), the intersection of G^{1}({(0, 0, −1), (0, 0, 1)}) with the plane

Π(τ) := {(x, y, t) ∈ ℍ | t = τ}, τ ∈ (−1, 1),

is a circle of radius r(τ) > 0 centered at (0, 0, τ). Pick any point p on the circle and denote by p^{∗} the reflection of p with respect to (0, 0, τ) in Π(τ). (0, 0, τ) ∈ G^{2}({(0, 0, −1), (0, 0, 1)}) now follows from Proposition 2.1 (iii).

3. Proof of the main results

3.1. Proof of Theorem 1.1. By Lemma 2.3, it is enough to show that C(A) = ℍ with A = {(0, 0, −1), (0, 0, 1)}. The proof is divided in three steps. First, we show that rotations that fix Z and reflections with respect to the (x, y)–plane map C(A) onto itself. Second, we prove that

\{(x, y, t) ∈ ℝ^{3} | 0 ≤ |t| < h(\sqrt{x^{2} + y^{2}})\} ⊆ C(A) (3.1)

for some non–increasing function h : [0, +∞) → [0, +∞]. The last step consists in showing that h is nowhere finite.

1. Let ϑ ∈ [0, 2π) and denote by R_{ϑ} ∈ R the rotation around Z with angle ϑ, as in (2.2). By Lemma 2.3, we have

R_{ϑ}(C(A)) = C(R_{ϑ}(A)) = C(A).

We denote by S : ℍ → ℍ, S(x, y, t) = (x, y, −t), the reflection with respect to the (x, y)–plane. We claim that S(C(A)) = C(A). By rotational symmetry of C(A), S(C(A)) = G(C(A)), where G(x, y, t) = (−x, y, −t). By Lemma 2.3

S(C(A)) = G(C(A)) = C(G(A)) = C(A).

2. We claim that for any pair of points (x, y, −t), (x, y, t) ∈ C(A) we have

{(x, y, t′) | 0 ≤ |t′| ≤ |t|} ⊆ C(A).

Notice first that C({(x, y, −t), (x, y, t)}) ⊆ C(A) and

C({(x, y, −t), (x, y, t)}) = C(l_{p} ∘ δ_{r}({(0, 0, −1), (0, 0, 1)}))

with p = (x, y, 0) and r = \sqrt{|t|}.

By Lemma 2.3 and Lemma 2.4 we obtain

{(x, y, t′) | 0 ≤ |t′| ≤ |t|} = l_{p} ∘ δ_{r}({(0, 0, t′) | 0 ≤ |t′| ≤ 1})
⊆ l_{p} ∘ δ_{r}(C({(0, 0, −1), (0, 0, 1)}))
= C(l_{p} ∘ δ_{r}({(0, 0, −1), (0, 0, 1)}))
= C({(x, y, −t), (x, y, t)}).

Now define the function h : [0, +∞) → [0, +∞] by letting

h(r) := sup{t ≥ 0 | (r, 0, t) ∈ C(A)}.

By the above considerations we have (3.1).

We claim that h is non–increasing. Otherwise we can find 0 ≤ r_{1} < r_{2} < +∞ such that 0 ≤ h(r_{1}) < h(r_{2}). Pick h(r_{1}) < t < h(r_{2}). Then by Proposition 2.1 (iii) the line segment [(−r_{2}, 0, t), (r_{2}, 0, t)] is contained in C(A), which contradicts h(r_{1}) < t.

3. Our goal in this last step is to prove that h is nowhere finite, which concludes the proof of Theorem 1.1. Assume by contradiction that there exists 0 < r < +∞ with 0 ≤ h(r) < +∞. Without loss of generality we can also assume h(r) > 0. Indeed, if h is nowhere positive and finite, then

r_{0} := inf{r ≥ 0 | h(r) = 0} < +∞.

It is easy to see that r_{0} > 0 and thus Z ⊆ C(A). For any t > 0, G^{2}({(0, 0, −t), (0, 0, t)}) ⊆ C(A) contains a Carnot-Carathéodory ball centered at the origin whose radius tends to +∞ as t tends to +∞ (note δ_{r}(G^{2}(B)) = G^{2}(δ_{r}(B)) for any set B ⊆ ℍ and any r > 0), which contradicts h(r) = 0 for r > r_{0}.

Let p^{−} = (r, 0, −h(r)), p^{+} = (r, 0, h(r)). By Proposition 2.1 (ii) (modulo left translation), there exists a geodesic γ : [0, d(p^{−}, p^{+})] → ℍ connecting p^{−} and p^{+} with

γ̇(0) = \frac{1}{\sqrt{2}} (X(p^{−}) + Y (p^{−})) = \frac{1}{\sqrt{2}} (∂_{x} + (∂_{y} − 2r∂_{t})) = \frac{1}{\sqrt{2}}(1, 1, −2r).

Hence, for some s > 0 we have γ(s) < −h(r) and γ_{1}^{2}(s) + γ_{2}^{2}(s) > r^{2}. But then the same is true for the geodesic γ̃ : [0, d((r, 0, −t), (r, 0, t))] → ℍ defined by

γ̃ = l_{p} ∘ δ_{λ} ∘ l_{p^{−1}} ∘ γ, λ = \sqrt{t/h(r)}, p = (r, 0, 0),

connecting (r, 0, −t) and (r, 0, t), provided that t ∈ (0, h(r)) is sufficiently close to h(r). This contradicts the fact that h is non–increasing.

3.2. Proof of Theorem 1.2. In the following, the map P : ℍ → ℍ defined by P (x, y, t) = (x, y, 0) is the orthogonal projection onto the (x, y)–plane.

The proof of Theorem 1.2 relies upon Theorem 1.1. We show that C({p_{1}, p_{2}, p_{3}}) contains a pair of points lying on the same vertical line. The following lemma is a key step towards this goal.

Lemma 3.1. Let A ⊆ ℍ. Suppose there exist q ∈ ℝ^{2}, a neighbourhood U of q in ℝ^{2} and a continuous function f : U → ℝ such that

Ã:= {(x, y, t) ∈ A | (x, y) ∈ U} = {(x, y, f(x, y)) | (x, y) ∈ U}.

Then A is not geodetically convex.

Proof. Without loss of generality we can assume that q = (0, 0). Suppose by contradiction that A is geodetically convex. Note that, by Proposition 2.1 (i) (modulo left translation), for a given pair of points in Ã there is a unique geodesic connecting them.

1. Choose r > 0 such that B := {(x, y) | x^{2} + y^{2} < r^{2}} ⊂⊂ U and define g : ∂B → ℝ by

g(x, y) := f(x, y) − f(−x, −y).

Since g(x, y) = −g(−x, −y), the continuity of g implies g(x, y) = 0 for some (x, y) ∈ ∂B, i.e. f(x, y) = f(−x, −y) for a pair of points (x, y), (−x, −y) in ∂B. Since A is geodetically convex, {(sx, sy, f(x, y)) | s ∈ [−1, 1]} ⊆ Ã. Hence f(sx, sy) = f(x, y) for all s ∈ [−1, 1]. By Lemma 2.3 we may assume that x > 0, y = 0 and f(0) = 0.

2. Let v ∈ S^{1} = {(x, y) ∈ ℝ^{2} | x^{2} + y^{2} = 1} ⊆ ℝ^{2}. Then the map s ↦ f(sv) from [−r, r] to ℝ is monotonic. Otherwise, by continuity of f, we could find s_{1} < s_{2} in [−r, r] such that f(s_{1}v) = f(s_{2}v), but either f(s_{i}v) > min{f(sv) | s ∈ [s_{1}, s_{2}]} or f(s_{i}v) < max{f(sv) | s ∈ [s_{1}, s_{2}]}, i = 1, 2, which is not possible because the image of the geodesic connecting (s_{1}v, f(s_{1}v)) and (s_{2}v, f(s_{2}v)) is a line segment contained in Ã.

3. The uniform continuity of f on compact subsets of U, the well–known estimate d(p_{1}, p_{2}) ≤ C_{K}|p_{1} − p_{2}|^{1/2} on compact subsets K of ℍ ≅ ℝ^{3} and the fact that the length of a geodesic is actually the Euclidean length of its projection onto the (x, y)–plane imply that the projection of the image of a geodesic connecting (x_{1}, y_{1}, f(x_{1}, y_{1})) and (x_{2}, y_{2}, f(x_{2}, y_{2})) is contained in an arbitrary small neighbourhood of (x_{i}, y_{i}), i = 1, 2, provided (x_{1}, y_{1}) and (x_{2}, y_{2}) are chosen sufficiently close to each other.

4. For 0 < ϵ < π/2, let

p_{0} := (r/2(cos(−ϵ), sin(−ϵ)), f(r/2(cos(−ϵ), sin(−ϵ)))),
p_{1} := (r/2(cos(ϵ), sin(ϵ)), f(r/2(cos(ϵ), sin(ϵ)))),
q_{0} := (r/2(cos(π − ϵ), sin(π − ϵ)), f(r/2(cos(π − ϵ), sin(π − ϵ)))),
q_{1} := (r/2(cos(π + ϵ), sin(π + ϵ)), f(r/2(cos(π + ϵ), sin(π + ϵ)))).

Let γ^{p} : [0, d(p_{0}, p_{1})] → ℍ and γ^{q} : [0, d(q_{0}, q_{1})] → ℍ be the geodesics satisfying γ^{p}(0) = p_{0}, γ^{p}(d(p_{0}, p_{1})) = p_{1}, γ^{q}(0) = q_{0} and γ^{q}(d(q_{0}, q_{1})) = q_{1}. If ϵ > 0 is chosen small enough, then (cf. 3.)

\begin{aligned}P ∘ γ^{p}([0, d(p_{0}, p_{1})]) & ⊆ (\{(x, y) ∈ B | 0 < x < r\}), and \\ P ∘ γ^{q}([0, d(q_{0}, q_{1})]) & ⊆ (\{(x, y) ∈ B | − r < x < 0\}). \end{aligned}

5. The horizontal plane at (x, 0, 0) is spanned by the vectors ∂_{x} and ∂_{y} − 2x∂_{t}. Now notice that there exists s_{p} ∈ (0, d(p_{0}, p_{1})) with the properties

γ_{2}^{p}(s_{p}) = γ_{3}^{p}(s_{p}) = 0, γ_{1}^{p}(s_{p}) > 0 and γ̇_{2}^{p}(s_{p}) > 0.

Indeed, γ^{p} must cross the (x, t)–plane, γ^{p}([0, d(p_{0}, p_{1})]) ⊆ Ã, f ≡ 0 on {(x, 0) | |x| ≤ r} (cf. 1.) and P ∘γ^{p}([0, d(p_{0}, p_{1})]) is a line segment or an arc of circle by Proposition 2.1. It follows that

γ̇_{3}^{p}(s_{p}) = 2γ̇_{1}^{p}(s_{p})γ_{2}^{p}(s_{p}) − 2γ̇_{2}^{p}(s_{p})γ_{1}^{p}(s_{p}) = −2γ̇_{2}^{p}(s_{p})γ_{1}^{p}(s_{p}) < 0.

Similarly, there exists s_{q} ∈ (0, d(q_{0}, q_{1})) with the properties

γ_{2}^{q}(s_{q}) = γ_{3}^{q}(s_{q}) = 0, γ_{1}^{q}(s_{q}) < 0, γ̇_{2}^{q}(s_{q}) < 0 and γ̇_{3}^{q}(s_{q}) < 0.

In particular, since γ^{p}([0, d(p_{0}, p_{1})]), γ^{q}([0, d(q_{0}, q_{1})]) ⊆ Ã and f is continuous, we can find 0 < δ < ϵ and 0 < ρ̃_{1}, ρ̃_{2} < r such that

f(ρ̃_{1}(cos(δ), sin(δ))) < 0 and f(ρ̃_{2}(cos(π + δ), sin(π + δ))) < 0.

Now, since f(0) = 0 and f is continuous on {ρ(cos(δ), sin(δ)) | − r ≤ ρ ≤ r}, there exist 0 < ρ_{1}, ρ_{2} < r such that t = f(ρ_{1}(cos(δ), sin(δ))) = f(ρ_{2}(cos(π + δ), sin(π + δ))) < 0. Since the image of the geodesic connecting (ρ_{1}(cos(δ), sin(δ)), t) and (ρ_{2}(cos(π + δ), sin(π + δ)), t) is a line segment contained in Ã, we get (0, 0, t) ∈ Ã, a contradiction.

Proof of Theorem 1.2. Given three points p_{1}, p_{2}, p_{3} ∈ ℍ not lying on the same geodesic, we have to show that C({p_{1}, p_{2}, p_{3}}) = ℍ. Without loss of generality, we can assume p_{1} = 0.

1. We claim that there exist two points q_{1}, q_{2} ∈ C({p_{1}, p_{2}, p_{3}}) such that q_{1} ≠ q_{2} and P (q_{1}) = P (q_{2}). Then Theorem 1.2 follows from Theorem 1.1. Assume by contradiction that no such pair of points exists. Then there is always a unique geodesic connecting any given two points in C({p_{1}, p_{2}, p_{3}}).

2. Consider the geodesic κ : [0, d(p_{2}, p_{3})] → ℍ satisfying κ(0) = p_{2} and κ(d(p_{2}, p_{3})) = p_{3}. For σ ∈ [0, d(p_{2}, p_{3})] let γ_{σ} : [0, d(p_{1}, κ(σ))] → ℍ be the unique geodesic such that γ_{σ}(0) = p_{1} and γ_{σ}(d(p_{1}, κ(σ))) = κ(σ). We show that if σ < τ, then γ_{σ} ∩ γ_{τ} = {p_{1}}. If the intersection is larger, let t_{1} := max{t ∈ [0, d(p_{1}, κ(σ))] | γ_{σ}(t) ∈ γ_{τ}} and let t_{2} be the unique element in [0, d(p_{1}, κ(τ))] with γ_{τ}(t_{2}) = γ_{σ}(t_{1}). By uniqueness of geodesics, t_{1} = t_{2} and γ_{σ}|[0, t_{1}] = γ_{τ}|[0, t_{2}]. It then follows from Proposition 2.1 (v) that either t_{1} = d(p_{1}, κ(σ)) or t_{2} = d(p_{1}, κ(τ)) and hence γ_{σ} ⊆ γ_{τ} or γ_{τ} ⊆ γ_{σ}. Consider for instance the case γ_{σ} ⊆ γ_{τ}. Clearly, κ([σ, τ]) ⊆ γ_{τ}. By Proposition 2.1 (v), it follows easily that κ(0) = p_{2} ∈ γ_{τ} and thus γ_{0}∪κ([0, τ]) ⊆ γ_{τ}. By our assumption on C({p_{1}, p_{2}, p_{3}}), some extension γ̃_{τ} of γ_{τ} must contain γ_{0} ∪ κ. Consequently p_{1}, p_{2}, p_{3} ∈ γ̃_{τ}, a contradiction.

3. Consider the open set

U = {(σ, s) ∈ ℝ^{2} | σ ∈ (0, d(p_{2}, p_{3})), s ∈ (0, d(p_{1}, κ(σ)))},

and the mapping F : U → ℝ^{2} given by

F (σ, s) := P (γ_{σ}(s)).

By 2., F is injective. Moreover, by Proposition 2.1 (vi), (σ, s) → γ_{σ}(s) is continuous, because the endpoint κ(σ) varies continuously. By the theorem on the invariance of domains – see e.g. Proposition 7.4 in A. Dold, “Lectures on Algebraic Topology”, Springer, 1972 – the mapping F is open. In particular the set V := F (U) is open and the inverse mapping F^{−1} : V → U is continuous. But then so is the function f : V → ℝ defined by

f(x, y) := g(F^{−1}(x, y)),

where g : U → ℝ is the third component of (σ, s) ↦ γ_{σ}(s). We have

{(x, y, t) ∈ C({p_{1}, p_{2}, p_{3}}) | (x, y) = V } = {(x, y, f(x, y)) | (x, y) ∈ V },

and by Lemma 3.1 the set C({p_{1}, p_{2}, p_{3}}) cannot be geodetically convex. This contradiction concludes the proof.

3.3. Proof of Corollary 1.3. We have to show that a function u : ℍ → ℝ which is convex along geodesics is constant. First we show that u must be constant on the vertical axis Z. Assume by contradiction this is not true.

Case 1: There exist three distinct points (0, 0, t_{1}), (0, 0, t_{2}), (0, 0, t_{3}) ∈ Z such that u(0, 0, t_{1}) ≤ u(0, 0, t_{2}) < u(0, 0, t_{3}). The set C := {p ∈ ℍ | u(p) < u(0, 0, t_{3})} is geodetically convex, because u is convex on geodesics. Moreover (0, 0, t_{1}), (0, 0, t_{2}) ∈ C and by Theorem 1.1 it follows that C = ℍ, contradicting (0, 0, t_{3}) ∉ C.

Case 2: u assumes exactly two values on the vertical axis (say 0 and 1), u(0, 0, t) = 0 for some t ∈ ℝ and u(p) = 1 for any p ≠ (0, 0, t) on the vertical axis (otherwise we are in Case 1). Consider two distinct geodesics γ and κ connecting (0, 0, t) and (0, 0, −t) (we can assume t ≠ 0 since geodesics are preserved by left translations). We have γ ∩ κ = {(0, 0, −t), (0, 0, t)}. By convexity of u on γ and κ, we can find p ∈ γ \ (γ ∩ κ) and q ∈ κ \ (γ ∩ κ) with u(p), u(q) < 1. The set C := {p′ ∈ ℍ | u(p′) < 1} is geodetically convex and contains (0, 0, t), p and q. Since these points do not lie on the same geodesic, Theorem 1.2 gives C = ℍ which contradicts (0, 0, t′) ∉ C when t′ ≠ t.

By left translation, the previous argument shows that u must be constant on any vertical line. Suppose now we could find two vertical lines v_{1} and v_{2} and c_{1} < c_{2}, such that c_{i}, i = 1, 2, is the value of u restricted to v_{i}. But then, if we choose two points on v_{1} sufficiently far apart, the union of images of geodesics connecting these two points will intersect v_{2}, which is impossible since by geodetic convexity we must have u ≤ c_{1} on this union.

References

[AR] Ambrosio, L., Rigot, S.: Optimal mass transportation in the Heisenberg group. J. Funct. Anal. 203, 261–301 (2004)

[B] Bellaïche, A.: The tangent space in sub-Riemannian geometry. In: Sub-Riemannian geometry, Prog. Math. 144, Birkhäuser Verlag, 1996, pp. 1–78

[BR] Balogh, Z.M., Rickly, M.: Regularity of convex functions on Heisenberg groups. Ann. Sc. Norm. Super. Pisa, Cl. Sci., IV. Ser., Vol. II, 847–868 (2003)

[DGN] Danielli, D., Garofalo, N., Nhieu, D.-M.: Notions of convexity in Carnot groups. Commun. Anal. Geom. 11, 263–341 (2003)

[G] Gaveau, B.: Principe de moindre action, propagation de la chaleur et estimées sous elliptiques sur certains groupes nilpotents. Acta Math. 139, 95–153 (1977)

[GM] Gutiérrez, C.E., Montanari, A.: Maximum and comparison principles for convex functions on the Heisenberg group. To appear in Commun. Partial Differ. Equations

[GT] Garofalo, N., Tournier, F.: Monotonicity and estimates of the supremum for Monge-Ampère measures in the Heisenberg group. Preprint

[K] Korányi, A.: Geometric properties of Heisenberg–type groups. Adv. Math. 56, 28–38, 1985

[LMS] Lu, G., Manfredi, J., Stroffolini, B.: Convex functions on the Heisenberg group. Calc. Var. Partial Differ. Equ. 19, 1–22 (2003)

[M] Monti, R.: Some properties of Carnot-Carathéodory balls in the Heisenberg group. Rend. Mat. Acc. Lincei, s.9, v.11, 155–167 (2000)

Dipartimento di Matematica Pura e Applicata, Via Belzoni 7, I–35131 Padova, Italia

  E-mail address: monti@math.unipd.it

Mathematisches Institut, Universität Bern, Sidlerstrasse 5, 3012 Bern, Schweiz

  E-mail address: matthieu.rickly@math-stat.unibe.ch