You can either select the tracks in the playlist you want removed and hit Delete or if you want to automate it edit the Smart Playlist settings to add a Last Played is not in the last X Days/Months/Weeks option. This will automatically removed a track after you've played in and limit when it can re-appear based on how long ago it was played.