Procedemos exactamente como has empezado, y luego utilizamos la fórmula producto-suma: $\sin(a) \sin(b) = \frac{1}{2} (\cos(a-b) - \cos(a+b))$ . Si $n \neq 1, 3$ :
$$\begin{aligned} I_n &= \displaystyle \int_0^{\pi} \sin^3(x) \sin(nx) \; \mathrm{d}x\\ &= \frac{1}{4} \displaystyle \int_0^{\pi} (3 \sin(x) - \sin(3x)) \sin(nx) \; \mathrm{d}x\\ &= \frac{1}{4} \displaystyle \int_0^{\pi} 3 \sin(nx) \sin(x) - \sin(nx) \sin(3x) \; \mathrm{d}x\\ &= \frac{1}{4} \displaystyle \int_0^{\pi} \frac{3}{2} \left ( \cos \left ((n-1) x \right ) - \cos \left ( (n+1) x \right ) \right ) - \frac{1}{2} \left ( \cos \left ( (n-3) x \right ) - \cos \left ( (n+3) x \right ) \right ) \; \mathrm{d}x\\ &= \frac{1}{8} \left [ \frac{3}{n-1} \sin \left ( (n-1) x \right ) - \frac{3}{n+1} \sin \left ( (n+1) x \right ) - \frac{1}{n-3} \sin \left ( (n-3) x \right ) + \frac{1}{n+3} \sin \left ( (n+3) x \right ) \right ]_0^{\pi}\\ &= 0 \end{aligned}$$
Así, si $n \neq 1, 3$ entonces la integral se evalúa como $0$ . Si $n=1$ tenemos:
$$\begin{aligned} I_1 &= \frac{1}{4} \displaystyle \int_0^{\pi} \frac{3}{2} \left ( \cos \left ((n-1) x \right ) - \cos \left ( (n+1) x \right ) \right ) - \frac{1}{2} \left ( \cos \left ( (n-3) x \right ) - \cos \left ( (n+3) x \right ) \right ) \; \mathrm{d}x\\ &= \frac{1}{8} \displaystyle \int_0^{\pi} 3 - 3\cos(2x)- \cos(-2x) + \cos(4x) \; \mathrm{d}x\\ &= \frac{1}{8} \left [3x - 3 \sin(2x) + \frac{3}{4} \sin(4x) \right ]_0^{\pi}\\ &= \frac{3\pi}{8} \end{aligned}$$
Si $n=3$ tenemos:
$$\begin{aligned} I_3 &= \frac{1}{4} \displaystyle \int_0^{\pi} \frac{3}{2} \left ( \cos \left ((n-1) x \right ) - \cos \left ( (n+1) x \right ) \right ) - \frac{1}{2} \left ( \cos \left ( (n-3) x \right ) - \cos \left ( (n+3) x \right ) \right ) \; \mathrm{d}x\\ &= \frac{1}{8} \displaystyle \int_0^{\pi} 3\cos(2x)-3\cos(4x)-1+\cos(6x) \; \mathrm{d}x\\ &= \frac{1}{8} \left [\frac{3}{2} \sin(2x) - \frac{3}{4}\sin(4x) - x + \frac{1}{6} \sin(6x) \right ]_0^{\pi}\\ &= -\frac{\pi}{8} \end{aligned}$$
Así, sobre todos los casos tenemos $$I_n = \begin{cases} 0 \quad &\text{if } n \neq 1, 3\\ \frac{3\pi}{8} \quad &\text{if } n = 1\\ -\frac{\pi}{8} \quad &\text{if } n = 3 \end{cases}$$