Rebuild for smaller images

Last in this series of attempted fixes, reverting to pip for most packages as pip/conda conflicts were not causing the issues, but keeping some of the changes