These two cameras are two entire different concepts in low light photography.
There are just as many differences as there are similarities.

The SD3500 is the next logical step of the SD series of cameras.

The SD4000 is a concept started by the Canon G11 and S90 cameras.
To get better low light performance they reduced the megapixels, installed a brighter lens, and added manual controls.

The pictures from the SD4000 are better than the pictures from the SD3500.
The main reason is the megapixel count.
The SD3500 has 14.1 megapixels and the SD4000 has 10 megapixels.
And they both use the same size sensor assembly.
That means they had to crowd 4 more megapixels onto the same size sensor assembly.

That results in more noise.
And looking closely at images from the SD3500 you can see the noise.
In order to reduce the noise, they get too aggressive with the noise reduction firmware in the camera and that results in loss of detail.

Last years SD980 has only 12.1 megapixels and takes better pictures than this years SD3500 (14.1 megapixels).

In my opinion, the step up from 12.1 to 14 megapixels has not been for the better.
That goes for all brands of cameras.

I suggest you read the reviews on the SD3500 and SD4000:

http://www.steves-digicams.com/camera-reviews/canon/powershot-sd3500-is/canon-powershot-sd3500-is-review.html

http://www.steves-digicams.com/camera-reviews/canon/powershot-sd4000-is/canon-powershot-sd4000-is-review.html

..