The 135mm f/2L lens may be a little long for portrait for the 40D, unless you like head shots a lot. It is great for indoor action photos without flash.
The 100mm f/2.8L macro is a very good lens.
But if you really want a portrait lens, then consider the 85mm f/1.2L II or the EF 70-200mm f/2.8L IS. The 24-105mm f/4L IS is also a good and versatile lens for close and group portrait shots. The 85mm f/1.8 is also good and less expensive.
I personally use the 70-200mm f/2.8L IS for close up head/shoulder portraits, and 24-105mm f/4L for general and group portrait shots on the 5D MkII. And on the 30D, I use the 17-55mm f/2.8 IS more often than the 24-105mm f/4L. I pick these lenses because they are very versatile and has good image qualities. I'm not a pro, most shots are candid shots, so a zoom lens is easier and more versatile for me.
Will be placing on a 40D looking to do portraits. Which would be the best investment? I know the ups and downs of each... I would just like some opinions. Thank you.