Should all lumps be removed?

Not necessarily. Some skin cancers can present as lumps but generally tend to have a change in the overlying skin as well. A lump that is under the skin which is quite small and moves on pressing it is commonly either a lipoma or a cyst. These are amenable to removal by surgery under a local anaesthetic.