A converse to Riemann's theorem on Jacobian varieties

Jacobians of curves have been studied a lot since Riemann’s theorem, which says that their theta divisor is a sum of copies of the curve. Similarly, for intermediate Jacobians of smooth cubic threefolds Clemens and Griffiths showed that the theta divisor is a sum of two copies of the Fano surface of lines on the threefold. We prove that in both cases these are the only decompositions of the theta divisor, extending previous results of Casalaina-Martin, Popa and Schreieder. Our ideas apply to a much wider context and only rely on the decomposition theorem for perverse sheaves and the representation theory of reductive groups.