We’re at war with whom, exactly?

A majority of those who support the war on terror have long noted this war is with the fringe element of radical Islam, not the entire Muslim world. The Bush administration has made this point in countless speeches on the issue, to the point of nausea whenever the President says “religion of peace”.
Yet what are we to think about this, that it is only “radical” Islam we are fighting, when those supposedly in the mainstream of the Muslim faith, knowing of the spreading of jihadist propoganda in their midst, and perhaps even knowing of jihadist plans of attack, fail to alert the authorities regarding these matters? Is that not tantamount to collusion, and if so, does this not mean we are finding ourselves at war with the entire Muslim world?