Don't seem to be able to delete those rejected photos.
Although on the menu bar says delete rejected photos, the pop up screen now says remove instead of delete. And true enough, the file was removed from LR database but not deleted from my harddisk.