The authors study the moduli space of trace-free irreducible rank 2 connections over a curve of genus 2 and the forgetful map towards the moduli space of underlying vector bundles (including unstable bundles), for which they compute a natural Lagrangian rational section. As a particularity of the genus $2$ case, connections as above are invariant under the hyperelliptic involution: they descend as rank $2$ logarithmic connections over the Riemann sphere. The authors establish explicit links between the well-known moduli space of the underlying parabolic bundles with the classical approaches by Narasimhan-Ramanan, Tyurin and Bertram. This allows the authors to explain a certain number of geometric phenomena in the considered moduli spaces such as the classical $(16,6)$-configuration of the Kummer surface. The authors also recover a Poincare family due to Bolognesi on a degree 2 cover of the Narasimhan-Ramanan moduli space. They explicitly compute the Hitchin integrable system on the moduli space of Higgs bundles and compare the Hitchin Hamiltonians with those found by van Geemen-Previato. They explicitly describe the isomonodromic foliation in the moduli space of vector bundles with $\mathfrak sl_2$-connection over curves of genus 2 and prove the transversality of the induced flow with the locus of unstable bundles.