We do have a horovod solver but its performance as never been fully tested - [x] test if the `SolverHorovod` is still working - [x] test on multiple CPU using MPI - [ ] test on multiple CPU/GPU