Non-smooth infinite dimensional group representations

Greg Hjorth

December 8, 1997

§0 When it’s bad it’s worse

It is known from [6] and [4] that the infinite dimensional irreducible representations of a countable group do not posses a nice Borel structure unless G is abelian-by-finite. Roughly speaking, the equivalence relation of conjugacy, or isomorphism, interprets the equivalence relation of eventual agreement on infinite sequences of integers, and thus is non-smooth: the irreducibles considered up to conjugacy do not form a standard Borel space, there is no Borel selector, there is no reasonable way to assign reals as complete invariants.

This note asks when the classification problem for the irreducibles is no worse than that of infinite sequences considered up to eventual agreement.

Surprsingly, the equivalence relation is always more complicated when non-smooth. Thus there is no reasonable way to assign complete invariants in sequences mod finite when G is not abelian-by-finite. Equivalently, the cosets of ℚ in ℝ can not provide complete invariants – that is to say, there is no Borel function θ from the space of irreducible representations to ℝ such that representations τ_{1} and τ_{2} are conjugate if and only if θ(τ_{1}) − θ(τ_{2}) ∈ ℚ. Worse: there is no countable Borel lacunary section (a Borel subset of the space of irreducibles meeting each equivalence class in exactly ℵ_{0} many points). We cannot assign countable sets of reals as complete invariants. And strongest of all – and this is the real theorem – for G not abelian-by-finite, we cannot complete a classification by countable discrete structures considered up to isomorphism.

§1 Some facts

From here on let G be a countable discrete group, H a separable infinite dimensional Hilbert space, U_{∞} = U(H) the unitary group of H – the group consisting of all bounded linear operators T : H → H with T^{∗} = T^{−1}.

1.1 Definition Define Irr(G, H) to be the space of irreducible representations – that is the set of all homorphisms τ : G → U_{∞} such that there are no non-trivial topologically closed, linearly closed, τ[G]- invariant subspaces of H.

For τ ∈ Irr(G, H) and T ∈ U_{∞}, define T · τ ∈ Irr(G, H) by

(T · τ)(g) = T ∘ τ(g) ∘ T^{−1}.

Let τ_{1} ≈ τ_{2} denote that there is some T ∈ U_{∞} with T · τ_{1} = τ_{2}.

Note that Irr(G, H) is a G_{δ} subset of H^{G}, and hence Polish (in the product topology). U_{∞} is a Polish group and the action of this group on Irr(G, H) is continuous.

The following is certainly implicit in [3].

1.2 Lemma (Effros, Mackey) Let τ_{1}, τ_{2} ∈ Irr(G, H), T_{i} ∈ U_{∞}, e, ê ∈ H, with

T_{i} · τ_{1} → τ_{2},
liminf_{j→∞}(⟨T_{j}(e), ê)⟩ ≠ 0.

Then τ_{1} ≈ τ_{2}. Proof. Without loss of generality there is a bounded linear operator T ∈ B(H) such that

T_{j}(e′) → T (e′)

weakly (in the sense that ⟨T_{j}(e′), e′′⟩ → ⟨T (e′), e′′⟩ all e′′ ∈ H) for all e′ ∈ H. Note then by replacing τ_{2} by suitable T′ · τ_{2} and T_{i} by T′T_{i} we may assume T_{j}(e) → δe weakly for some δ > 0. Claim(1): For all e′ ∈ H,

T_{j}^{∗}(e′) → T^{∗}(e′).

Proof of claim. Since for all e′, e′′ ∈ H we have that

⟨e′, T^{∗}(e′′)⟩ = ⟨T (e′), e′′⟩
= lim_{i→∞}⟨T_{i}(e′), e′′⟩
= lim_{i→∞}⟨e′, T_{i}^{∗}(e′′)⟩.

(□Claim(1))

Claim(2): For all g ∈ G

T ∘ τ_{1}(g) = τ_{2}(g) ∘ T.

Proof of claim: Since for all e′, e′′ ∈ H

⟨T ∘ τ_{1}(g)(e′), e′′⟩ = lim⟨T_{i} ∘ τ_{1}(g)(e′), e′′⟩
= lim_{i→∞}⟨τ_{2}(g) ∘ T_{i}(e′), e′′⟩ = lim_{i→∞}⟨T_{i}(e′), τ_{2}(g)^{∗}(e′′)⟩
⟨T (e′), τ_{2}(g)^{∗}(e′′)⟩
= ⟨τ_{2}(g) ∘ T (e′), e′′⟩.
(□Claim(2))

Claim(3): For all g ∈ G

T^{∗} ∘ τ_{2}(g) = τ_{1}(g) ∘ T^{∗}.

Proof of claim: Since for all e′ ∈ H and j ∈ ℕ we have

T_{j}^{∗} ∘ τ_{2}(g)(T_{j}(e′)) = T_{j}^{∗} ∘ T_{j} ∘ τ_{1}(e′)
= τ_{1}(e′) = τ_{1} ∘ T_{j}^{∗}(T_{j}(e′)),

we can use claim(1) to just repeat the proof of claim(2). (□Claim(3)) Claim(4): For all g ∈ G

T^{∗} ∘ T ∘ τ_{1}(g) = τ_{1}(g) ∘ T^{∗} ∘ T.

Proof of claim: Follows by claim(2) and claim(3). (□Claim(4))

So now let B ⊂ B(H) be the C^{∗}-algebra generated by the operator T^{∗} ∘ T. Since T^{∗} ∘ T is self adjoint we have that the polynomials in T^{∗} ∘ T are dense in B, and hence that every τ_{1}(g) commutes with every element of B. Following 3.5, 3.6, 3.11VIII of [2], if we let M_{0} be the orthogonal complement of the kernel of T and let M_{1} be the range of T, we may find an isometry

U : M_{0} → M_{1}

and positive S ∈ B with

U ∘ S = T
S^{2} = T ∗ T.

Since every element of B commutes with τ_{1}(g) we have that

U ∘ τ_{1}(g) ∘ S = U ∘ S ∘ τ_{1}(g)

= T ∘ τ_{1}(g) = τ_{2}(g) ∘ T = τ_{2}(g) ∘ U ∘ S

and thus U ∘τ_{1}(g) equals τ_{2}(g)∘U on range of S. Since RanS is τ_{1}[G]-invariant we have RanS = ∅, H by assumption of irreducibility. So we will be done once we see that S is non-degenerate. However, T ∗ T ≠ 0 by assumption

⟨T ∗ ∘T (e), e⟩ = ⟨T (e), T e⟩
= ⟨δe, δe⟩ =≠ 0.
□

1.3 Corollary Every ≈-equivalence calss is F_{σ}.

§2 A lemma

From [5]:

2.1 Theorem Let G_{0} be a Polish group. Let S_{∞} be the infinite symmetric group, let G_{1} be a closed subgroup of S_{∞}. Suppose X, Y are Polish spaces on which G_{0} and G_{1} respectively act continuously. Suppose that

(i) every G_{0}-orbit in X is dense; (ii) every G_{0}-orbit in X is meager; (iii) for all x ∈ X, the local orbits of x are somewhere dense; that is to say, if V is an open neighbourhood of 1_{G_{0}}, U is an open set containing x, and if O(x, U, V) is the set of all x̂ ∈ [x]_{G_{0}} such that there is a finite sequence (x_{i})_{i≤k} ⊂ U such that x_{0} = x, x_{k} = x̂, and each x_{i+1} ∈ V · x_{i}, then the closure of O(x, U, V) contains an open set.

Then there is no Borel function

θ : X → Y

such that for all x_{1}, x_{2} ∈ X

∃g ∈ G_{0}(g · x_{1} = x_{2}) ⇔ ∃g′ ∈ G_{1}(g′ · θ(x_{1}) = θ(x_{2})).

2.2 Lemma Let G be a countable group that is not abelian-by-finite – that is to say, there is no abelian subgroup G_{0} < G with finite index (|{gG_{0} : g ∈ G}| < ℵ_{0}). Then there is a U_{∞}-invariant G_{δ} set X on which (i), (ii), and (iii) of 2.1 hold – that is to say, every U_{∞}-orbit in X is dense, meager, and has all its local orbits somewhere dense. Proof. By [6], [4] there is no Borel function θ_{0} :Irr(G, H) → ℝ such that for all τ_{1}, τ_{2} ∈ Irr(G, H)

τ_{1} ≈ τ_{2} ⇔ θ_{0}(τ_{1}) = θ_{0}(τ_{2}).

Therefore in particular, the closure of an orbit does not provide a complete invariant, and so we may find σ_{1}, σ_{2} ∈ Irr(G, H) such that

σ_{1} ≉ σ_{2}

but

\overline{U_{∞} · σ_{1}} = \overline{U_{∞} · σ_{2}}.

Let C = \{τ ∈ Irr(G, H) : \overline{U_{∞} · τ} = \overline{U_{∞} · σ_{1}}\}. C is a G_{δ} subset of Irr(G, H) and hence Polish. Now on we shall restrict our attention to the Polish space C. From 1.3 we have every orbit F_{σ},

and thus, by the Baire category theorem no orbit in C is G_{δ}. So by 2.2.3 [1], every orbit in C is meager. By definition every orbit is dense, so we are just left to show every local orbit somewhere dense in the sense of 2.1(iii) above.

Let U ⊂ C be (relatively) open. Let V ⊂ U_{∞} be an open symmetric neighbourhood of 1_{U_{∞}}. Let τ_{1} ∈ U. Let {e_{i} : i ∈ ℕ} be an orthonormal basis for H. Let δ > 0, A ⊂ G of (finite) size N, and n ∈ ℕ be such that

{τ ∈ C : ⟨τ(g)(e_{i}), e_{j}⟩ − ⟨τ_{1}(g)(e_{i}), e_{j}⟩| < δ for all i, j ≤ n, g ∈ A} ⊂ U

and let

U_{0} = {τ ∈ C : ⟨τ(g)(e_{i}), e_{j}⟩ − ⟨τ_{1}(g)(e_{i}), e_{j}⟩| < δ/2 for all i, j ≤ n, g ∈ A} ⊂ U

and

{T ∈ U_{∞} : ⟨T (e_{i}), e_{i}⟩ > 1 − δ all i ≤ n} ⊂ V.

Choose U_{1} ⊂ U_{0} non-empty. We seek O(τ_{1}, U, V)∩U_{1} ≠ ∅. Choose τ_{2} ∈ U_{1} with τ_{1} ≉ τ_{2}. Choose T_{i} ∈ U_{∞} with

T_{i} · τ_{1} → τ_{2}.

Let {f_{1}, ..., f_{k}} be an orthonormal basis for the subbasis generated by {(τ_{1}^{∗}(g))(e_{i}), (τ_{1}(g))(e_{i}), e_{i} : g ∈ A, i ≤ n}. By 1.2, for all j_{1}, j_{2} ∈ ℕ there is some ϵ > 0 such that there are only finitely many i ∈ ℕ with

|⟨e_{j_{1}}, T_{i}(f_{j_{2}})⟩ > |ϵ.

Thus for δ_{0} > 0 arbitrarily small we may find i such that for all j_{1} ≤ n, j_{2} ≤ k and g ∈ A

|⟨e_{j_{1}}, T_{i}(f_{j_{2}})⟩| < δ_{0}/k,
T_{i} · τ_{1} ∈ U_{1}.

Claim: Let f′_{1}, ..., f′_{l} be a set of unit vectors in H, and let H_{0} be a finite dimensional subspace of H with orthonormal basis f̂_{1}, ..., f̂_{k′}. If for each i ≤ k′

∑_{j≤l} |⟨f′_{i}, f̂_{j}⟩| < ϵ,

then there is T ∈ U_{∞} such that for each ⟨f′_{i}, T (f′_{i})⟩ is bigger than 1 − ϵ and each |⟨f̂_{i}, T (f′_{i})⟩| = 0.

Proof of claim. Let {f̂_{i} : i ∈ ℕ} be some orthonormal basis of H extending the orthonormal basis of H_{0} and such that {f̂_{i} : i ≤ k′ +l} provides a basis for the subspace generated by H_{0} ∪{f′_{1}, ..., f′_{k′}}. Then choose T so that it interchanges {f̂_{1}, ..., f̂_{k′}} with {f̂_{1+l+k′}, ..., f̂_{k′+l+k′}} and use that

∑_{j∈ℕ} |⟨f′_{i}, f̂_{j}⟩|^{2} = 1.

(□Claim)

Applying the claim to the orthonormal set {T_{i}(f_{j}) : j ≤ k} we can find T′ ∈ U_{∞} such that (i) for all j_{1}, j_{2} ≤ n and g ∈ A

⟨T′(e_{j_{1}}), T_{i}((e_{j_{2}}))⟩ = 0,
∴ ⟨T_{i}^{−1}(T′(e_{j_{1}})), e_{j_{2}}⟩ = 0,
∴ ⟨τ_{1}(g)(T_{i}^{−1}(T′(e_{j_{1}}))), τ_{1}(g)(e_{j_{2}})⟩ = 0,
⟨T′(e_{j_{1}}), T_{i}(τ_{1}(g)(e_{j_{2}}))⟩ = 0,
∴ ⟨T_{i}^{−1}(T′(e_{j_{1}})), τ_{1}(g)(e_{j_{2}})⟩ = 0,
⟨T′(e_{j_{1}}), T_{i}(τ_{1}^{∗}(g)(e_{j_{2}}))⟩ = 0
∴ ⟨τ_{1}(g) ∘ T_{i}^{−1} ∘ T′(e_{j_{1}}), e_{j_{2}}⟩ = 0;

(ii) for all j ≤ n

|⟨T′(e_{j}), e_{j}⟩| > 1 − δ_{0}.

From (ii) we obtain that

(iii) for all j ≤ n

|⟨(T′)^{−1}(e_{j}), e_{j}⟩| > 1 − δ_{0}.

and so (T′)^{−1} ∈ V.

For future notation, let τ_{3} = (T′)^{−1} ∘ T_{i} · (τ_{1}). Now note that for all j_{1}, j_{2} ≤ n

⟨τ_{3}(g)(e_{j_{1}}), e_{j_{2}}⟩
= ⟨((T′)^{−1} · (T_{i} · (τ_{1})))(g)(e_{j_{1}})), e_{j_{2}}⟩
= ⟨((T′)^{−1}(T_{i} · (τ_{1})))(g))(T′(e_{j_{1}}))), e_{j_{2}}⟩
= ⟨(T_{i} · τ_{1}(g))(T′(e_{j_{1}})), T′(e_{j_{1}})⟩ =
⟨(T_{i} · τ_{1}(g))(δ_{1}e_{j_{1}}), δ_{2}e_{j_{2}}⟩ + ⟨(T_{i} · τ_{1}(g))(δ_{3}f), δ_{2}e_{j_{2}}⟩
+⟨(T_{i} · τ_{1}(g))(δ_{1}e_{j_{1}}), δ_{4}f′⟩ + ⟨(T_{i} · τ_{1}(g))(δ_{3}f), δ_{4}f′⟩

where

1 > δ_{1} = |⟨T′(e_{j_{1}}), e_{j_{1}})⟩| > 1 − δ_{0},
1 > δ_{2} = |⟨T′(e_{j_{2}}), e_{j_{2}})⟩|√(1 − δ_{0}),

and

(δ_{1})^{2} + (δ_{3})^{2} = 1,

(δ_{2})^{2} + (δ_{4})^{2} = 1
f⊥e_{j_{1}},
f′⊥e_{j_{2}}.

Thus, since each T_{i} · τ_{1}(g) is an isometry, for suitably small δ_{0} we can obtain

⟨((T′)^{−1} · (T_{i} · (τ_{1})))(g)(e_{j_{1}})), e_{j_{2}}⟩ − ⟨T_{i} · τ_{1}(g))((e_{j_{1}})), e_{j_{2}}⟩| < δ/2,

and thus τ_{3} = (T′)^{−1} ∘ T_{i} · (τ_{1}) ∈ U. Since T_{i} · (τ_{1}) ∈ O((T′)^{−1} ∘ T_{i} · (τ_{1}), U, V), it suffices to show that (T′)^{−1} ∘ T_{i} · (τ_{1}) ∈ O(τ_{1}, U, V).

Let {f_{j} : j ∈ ℕ} be an orthonormal basis extending {e_{j}, (T_{j})^{−1} ∘ T′(e_{j}) : j ≤ n}. For φ ∈ [0, 2π] define S_{φ} ∈ U_{∞} by the requirement that for all j ≤ n

S_{φ}(e_{j}) = cos(φ)(e_{j}) − sin(φ)((T_{i})^{−1} ∘ T′(e_{j})),
S_{φ}((T_{i})^{−1} ∘ T′(e_{j})) = sin(φ)(e_{j}) + cos(φ)((T_{i})^{−1} ∘ T′(e_{j})),

while for f_{j} outside {e_{j}, (T_{i})^{−1} ∘ T′(e_{j}) : j ≤ n} we just have

S_{φ}(f_{j}) = f_{j}.

Since (S_{π/2})^{−1}(e_{j}) = (T_{i})^{−1} ∘ T′(e_{j}) for each j ≤ n it follows that (T′)^{−1} ∘ T_{i} ∘ (S_{π/2})^{−1} ∈ V. Thus if we can only show S_{π/2} · τ_{1} ∈ O(τ_{1}, U, V) then we obtain τ_{3}(=_{df} (T′)^{−1} ∘ T_{i} · (τ_{1})) is in O(S_{π/2} ·τ_{1}, U, V), and hence T_{i} ·τ_{1} ∈ O(τ_{3}, U, V) and the transitivity of this relation O(·, U, V) yields T_{i} · τ_{1} ∈ O(τ_{1}, U, V) as required.

Then for all j_{1}, j_{2} ≤ n, g ∈ A we have

|⟨τ_{1}(g)(S_{φ}^{−1}(e_{j_{1}})), S_{φ}^{−1}(e_{j_{2}})⟩|

= |⟨τ_{1}(g)(cos(−φ)(e_{j_{1}}) − sin(−φ)((T_{i})^{−1} ∘ T′(e_{j_{1}})), cos(−φ)(e_{j_{2}}) − sin(−φ)((T_{i})^{−1} ∘ T′(e_{j_{2}})⟩|

= |⟨cos^{2}(−φ)⟨τ_{1}(g)(e_{j_{1}}), e_{j_{2}}⟩ + sin^{2}(−φ)⟨τ_{1}(g)((T_{i})^{−1} ∘ T′(e_{j_{1}})), (T′)^{−1} ∘ T_{i}(e_{j_{2}})⟩|

by orthogonality of the sets {e_{j}, (τ_{1}(g))(e_{j}) : j ≤ n, g ∈ A} and {(T_{i})^{−1}∘T′(e_{j}), (τ_{1}(g))(T_{i})^{−1}∘T′(e_{j})) : j ≤ n, g ∈ A} as expressed at (i) above. Thus

|⟨(S_{φ} · (τ_{1}))(g)(e_{j}), e_{i}⟩ − ⟨(τ_{1}(g))(e_{j_{1}}), e_{j_{2}}⟩| =
= |⟨cos^{2}(−φ)⟨τ_{1}(g)(e_{j_{1}}), e_{j_{2}}⟩ + sin^{2}(−φ)⟨τ_{1}(g)((T_{i})^{−1} ∘ T′(e_{j_{1}})), (T_{i})^{−1} ∘ T′(e_{j_{2}})⟩
−⟨(τ_{1})(g)(e_{j_{1}}), e_{j_{2}}⟩|

But this final inequality is bounded by an expression of the form

λ|⟨τ_{1}(g)((T_{i})^{−1} ∘ T′(e_{j_{1}})), (T_{i})^{−1} ∘ T′(e_{j_{2}})⟩ − ⟨(τ_{1})(g)(e_{j_{1}}), e_{j_{2}}⟩|
+(1 − λ)|⟨τ_{1}(g)(e_{j_{1}}), e_{j_{2}}⟩ − ⟨(τ_{1})(g)(e_{j_{1}}), e_{j_{2}}⟩|,

where λ ∈ [0, 1], and hence must be less than or equal to the bound δ obtained at the extreme points. Thus each S_{φ} · (τ_{1}) is in U, and so we have traced out a continuous path of group elements connecting τ_{1} to S_{π/2} · τ_{1} inside U, and are therefore done. □

The further non-classifiability results alluded to in the introduction are now a consequence of 2.1 and standard facts from the theory of equivalence relations – as may be found in [1].

References

[1] H. Becker and A.S. Kechris, The descriptive set theory of Polish group actions, London Mathematical Society Lecture Notes Series, Cambridge, 1996.

[2] J.B. Conway, A course in functional analysis, Graduate Texts in Mathematics 96, Springer-Verlag, New-York, 1990.

[3] E. Effros, Transformation groups and C^{∗}-algebras, Annals of Mathematics, ser 2, vol. 81(1975), pp. 38-55.

[4] J. Glimm, Locally compact transformation groups, Transactions of the American Mathematical Society, vol. 101(1961), pp. 124-138.

[5] G. Hjorth, Classification and orbit equivalence relations, unpublished manuscript, UCLA 1997.

[6] E. Thoma, Eine Charakterisierung diskreter Gruppen vom type I, Inventiones mathematique, vol. 6(1968), pp. 190-196.

6363 MSB

Mathematics

UCLA

CA90095-1555

greg@math.ucla.edu