$\displaystyle J=\int_0^1 \frac{\ln^2 x}{\sqrt{x(1-x)}}\,dx$
Realizar el cambio de variable $x=\sin^2 \theta$,
$\displaystyle J=8\int_0^{\frac{\pi}{2}}\ln^2(\sin \theta)\,d\theta$
Observar que,
$\displaystyle K=\int_0^{\frac{\pi}{2}}\ln^2(\sin \theta)\,d\theta=\int_0^{\frac{\pi}{2}}\ln^2(\cos \theta)\,d\theta$
(realizar el cambio de variable $\displaystyle y=\frac{\pi}{2}-x$ )
$\begin{align} K&=\int_0^{\frac{\pi}{2}}\ln^2(\tan \theta\cos \theta)\,d\theta\\
&=\int_0^{\frac{\pi}{2}} \left(\ln\left(\tan \theta\right)+\ln\left(\cos\theta\right) \right)^2\,d\theta\\
&=\int_0^{\frac{\pi}{2}}\ln^2\left(\tan \theta\right)\,d\theta+K+2\int_0^{\frac{\pi}{2}} \ln\left(\tan \theta\right)\ln\left(\cos \theta\right)\,d\theta\\
&=\int_0^{\frac{\pi}{2}}\ln^2\left(\tan \theta\right)\,d\theta+K+2\int_0^{\frac{\pi}{2}}\Big(\ln(\sin \theta)-\ln(\cos \theta)\big)\ln(\cos \theta)\,d\theta\\
&=\int_0^{\frac{\pi}{2}}\ln^2\left(\tan \theta\right)\,d\theta-K+2\int_0^{\frac{\pi}{2}}\ln(\sin \theta)\ln(\cos \theta)\,d\theta
\end{align}$
Por lo tanto,
$\displaystyle 2K=\int_0^{\frac{\pi}{2}}\ln^2\left(\tan \theta\right)\,d\theta+2\int_0^{\frac{\pi}{2}}\ln(\sin \theta)\ln(\cos \theta)\,d\theta$
pero,
$\begin{align}4\int_0^{\frac{\pi}{2}}\ln(\sin \theta)\ln(\cos \theta)\,d\theta&=\int_0^{\frac{\pi}{2}} \Big(\big(\ln(\sin \theta)+\ln(\cos \theta)\big)^2-\big(\ln(\sin \theta)-\ln(\cos \theta)\big)^2\Big)\,d\theta\\
&=\int_0^{\frac{\pi}{2}}\ln^2(\sin \theta\cos \theta)\,d\theta-\int_0^{\frac{\pi}{2}}\ln^2(\tan \theta)\,d\theta\end{align}$
Por lo tanto,
$\displaystyle 2K=\frac{1}{2}\int_0^{\frac{\pi}{2}}\ln^2\left(\tan \theta\right)\,d\theta+\frac{1}{2}\int_0^{\frac{\pi}{2}}\ln^2(\sin \theta\cos \theta)\,d\theta$
Pero,
$\displaystyle L=\int_0^{\frac{\pi}{2}}\ln^2(\sin \theta\cos \theta)\,d\theta=\int_0^{\frac{\pi}{2}}\ln^2\left(\frac{1}{2}\sin(2\theta)\right)\,d\theta$
Realizar el cambio de variable $\displaystyle x=2\theta$,
$\begin{align}L&=\frac{1}{2}\int_0^{\pi}\ln^2\left(\frac{1}{2}\sin\theta\right)\,d\theta\\
&=\frac{1}{2}\int_0^{\frac{\pi}{2}}\ln^2\left(\frac{1}{2}\sin\theta\right)\,d\theta+\frac{1}{2}\int_{\frac{\pi}{2}}^{\pi}\ln^2\left(\frac{1}{2}\sin\theta\right)\,d\theta\\
\end{align}$
En el último integral realizar el cambio de variable $\displaystyle x=\pi-\theta$,
$\begin{align}
L&=\int_0^{\frac{\pi}{2}}\ln^2\left(\frac{1}{2}\sin\theta\right)\,d\theta\\
&=\int_0^{\frac{\pi}{2}}\left(\ln(\sin\theta)-\ln 2\right)^2\,d\theta\\
&=K-2\ln 2\int_0^{\frac{\pi}{2}}\ln(\sin \theta)\,d\theta+\frac{1}{2}\pi \ln^2 2
\end{align}$
y es bien kwown que,
$\displaystyle \int_0^{\frac{\pi}{2}}\ln(\sin \theta)\,d\theta=-\frac{1}{2}\pi\ln 2$
Por lo tanto,
$\displaystyle L=K+\frac{3}{2}\pi \ln^2 2$
Por lo tanto,
$\begin{align}K&=\frac{1}{3}\int_0^{\frac{\pi}{2}}\ln^2\left(\tan\theta\right)\,d\theta+\frac{1}{2}\pi\ln^2 2\\
\end{align}$
Realizar el cambio de variable $\displaystyle y=\tan x$,
$\begin{align}
K&=\frac{1}{3}\int_0^{\infty}\frac{\ln^2 x}{1+x^2}\,dx+\frac{1}{2}\pi\ln^2 2\\
&=\frac{1}{3}\int_0^{1}\frac{\ln^2 x}{1+x^2}\,dx+\frac{1}{3}\int_1^{\infty}\frac{\ln^2 x}{1+x^2}\,dx+\frac{1}{2}\pi\ln^2 2\\
\end{align}$
En el último integral realizar el cambio de variable $\displaystyle y=\frac{1}{x}$,
$\begin{align}
K=\frac{2}{3}\int_0^{1}\frac{\ln^2 x}{1+x^2}\,dx+\frac{1}{2}\pi\ln^2 2\\
\end{align}$
Por lo tanto,
$\begin{align}
\boxed{J=\frac{16}{3}\int_0^{1}\frac{\ln^2 x}{1+x^2}\,dx+4\pi\ln^2 2}\\
\end{align}$
pero,
$\displaystyle \int_0^{1}\frac{\ln^2 x}{1+x^2}\,dx=2\beta(3)$
(Beta de la función de Dirichlet, https://en.wikipedia.org/wiki/Dirichlet_beta_function )
y es bien sabido que,
$\displaystyle \beta(3)=\frac{\pi^3}{32}$
(véase, por ejemplo, https://math.stackexchange.com/a/613341/186817 )
Por lo tanto,
$\boxed{\displaystyle J=\frac{\pi}{3}+4\pi\ln^2 2}$
PS:
otro enlace útil,
Calcular esta integral, $\int_0^{\infty}\frac{\ln^2x}{1+x^2}dx$