Any non-volatile storage medium that is not directly accessible to the processor. Memory directly accessible to the processor includes main memory, cache and the CPU registers. Secondary storage includes hard drives, magnetic tape, CD-ROM, DVD drives, floppy disks, punch cards and paper tape.
Secondary storage devices are usually accessed via some kind of controller. This contains registers that can be directly accessed by the CPU like main memory ("memory mapped"). Reading and writing these registers can cause the device to perform actions like reading a block of data off a disk or rewinding a tape. See also DMA.