Yes, I can erase if I either hit E (keyboard or express key on the tablet) or if I select the eraser.
I've indirectly solved the issue by choosing to put Alt and Right Click on the pen instead of Erase/Pen and Right Click.
I'd still be interested in figuring this issue out though.