Born in Freiburg, Germany, moved to Milwakee, WI at the age of 1 year. In 1919, Holty studied briefly at the school of the ArtInstitute of Chicago. Shortly thereafter, he left for New York, where he enrolled at Parsons School of Design and subsequently at the National Academy of Design. In 1923, he returned to Milwaukee to pursue a career as a portraitist. Three years later he left with his new bride for Munich, planning to study at the Royal Academy. Instead, he ran into his friend Vaclav Vytlacil, who convinced Holty to join Hans Hofmann's school.