That's not too easy.

You can set the email account to "leave messages on server" on both machines, but then you have to go to their webmail to clear that regularly.
When it's set on none of them, like now, it's first-come-first-served: the one that happens to ask first (and maybe it's set to once in 10 minutes independently) gets the message.
When you set it on only ONE, the OTHER PC gets ALL messages and THAT PC only part.

In other words, you've discovered why webmail (gmail, hotmail etc) is so popular by people using more than one PC.

As an alternative, consider the use of "hosted Exchange" as a central place to store your data. That's MS Outlook compatible.