A set of electrical conductors (wires, PCB tracks or connections in an integrated circuit) connecting various "stations", which can be functional units in a computer or nodes in a network. A bus is a broadcast channel, meaning that each station receives every other station's transmissions and all stations have equal access to the bus.
The term is almost certainly derived from the electrical engineering term "bus bar" - a substantial, rigid power supply conductor to which several connections are made. This was once written "'bus bar" as it was a contraction of "omnibus bar" - a connection bar "for all", by analogy with the passenger omnibus - a conveyance "for all".
More on derivation.
There are busses both within the CPU and connecting it to external memory and peripheral devices. The data bus, address bus and control signals, despite their names, really constitute a single bus since each is useless without the others.
The width of the data bus is usually specified in bits and is the number of parallel connectors. This and the clock rate determine the bus's data rate (the number of bytes per second which it can carry). This is one of the factors limiting a computer's performance. Most current microprocessors have 32-bit busses both internally and externally. 100 or 133 megahertz bus clock rates are common. The bus clock is typically slower than the processor clock.
Some processors have internal busses which are wider than their external busses (usually twice the width) since the width of the internal bus affects the speed of all operations and has less effect on the overall system cost than the width of the external bus.