Aicher was born in Ulm, in the south-western state of Baden-Württemberg, on May 13, 1922. In 1937 he was arrested for refusing to join the Hitler Youth and then he failed his college entrance examination in 1941. He was then drafted into the German army to fight in World War II, though he tried to avoid it. He deserted the army and went into hiding in the Scholls' house in Wutach in 1945. In 1946, after the end of the war, Aicher began studying sculpture at the Academy of Fine Arts Munich. In 1947, he opened his own studio in Ulm. He married Inge Scholl in 1952.
In 1966 Aicher was asked by the organisers of the 1972 Summer Olympics in Munich to become the Olympic Games' lead designer. He was asked to create a design for the Olympics that complemented the architecture of the newly built stadium in Munich designed by Günther Behnisch. Aicher consulted with Masaru Katsumie, who had designed the previous 1964 Tokyo Olympic Games.