How do I tell a friend he's not invited?

I'm going on a camping trip this weekend with some of my family (brother and sister and their significant others) and my friend said I should bring him along. I'd prefer he didn't come. I've actually been trying to distance myself from him, but we ride the bus to work together. He's a bit of an emotional vampire. I'm moving in a few weeks anyway and will see him only rarely at work. I don't think it would be necessary to bash him when I'm not going to see him much anymore anyway. Anyone with a polite idea to let him down? I thought about saying there won't be enough food or my nephews would be annoying, but those seem really weak and transparent. Maybe say my brother is pretty exclusive, which he is. Anyways he just threw this at me and I leave tomorrow. Please help me quick.