The two key issues are:
1.) Quality of the output. Check reviews of the make/model with the integrated chip and consider the desired TV's size/resolution as performance varies widely.
2.) Shared RAM. Remember that the RAM used for your graphics system is not available for general use by Windows and installed applications, so shop accordingly.
P.S. Windows 7 Starter and Windows 7 Home Basic are limited, but such are rarely found outside of netbooks and 'developing' countries. Thus, such should not be a problem.
I am looking at buying a new laptop (core i5 or i7 cpu) and am wondering if I need to buy one with a dedicated graphics card. I don't play any computer games, but I would like to connect my laptop to my tv to watch movies and some tv shows through my computer. I know just about all computers can do this, but I was under the impression that not all could do it with Windows 7.