The scaling of decoherence rates with the number of q-bits is studied for a simple quantum computer model. Two state q-bits are localised around well-separated positions via trapping potentials, but vibrational motion of q-bits centre of mass motion occurs. Coherent one and two q-bit gating processes are controlled by external classical fields and facilitated by a high Q cavity mode. Decoherence due to q-bit and cavity mode coupling to a bath of spontaneous emission modes, cavity decay modes and to the vibrational modes is treated. A non-Markovian treatment of the short time behaviour of the fidelity is presented, enabling time scales for decoherence to be determined, together with their dependence on q-bit number for the case where the q-bit/cavity mode system is in a pure state and the baths are in thermal states.