Bullets aren't harmless, you crazy guy.
buying them is harmless. making them is harmless. i bet you're thinking of shooting them at other people, which i believe should not be legal. fortunately, that activity is pretty conspicuous, and really isn't that easy either.
I think this is getting into semantics. If you think that bullets, by their nature, "are harmless," i would imagine that your definition of "harmless" includes far more things than most others' would - which is okay, but important to recognize that you're opinion probably isn't widely held. That's why regulations on bullets exist.
I understand your point of view and I recognize that it's consistent and rational, but our state operates functionally different than the one for which you're advocating. The fact is, our laws prohibit certain activities, that in and of themselves, do no harm to others, are easy to do, and are inconspicuous. Given your argument, I'd imagine you think that those activities (like the possession of heroin, for example) should be legal, but they're not.
Also, we, as a country, have decided that ammunition, specifically, isn't harmless. That's why we regulate it.