Wittgenstein and Quine are usually regarded as standing in opposition to each other, as the former is an antinaturalist who defends the separation of science and philosophy while the latter naturalistically insists on the continuity of science and philosophy. On the other hand, they both claim to provide an analysis of our conceptual scheme “from within” and reject the very idea of a first philosophy. Wittgenstein’s investigations take normativity into account, whereas Quine appears to be uninterested. Can naturalism, in some form, countenance normativity or is it bound to be reductionist about it? I suggest that Wittgenstein’s conceptual analysis should be read as exemplifying a form of non-reductionist ontological naturalism that sees man as a natural species, while admitting that the normativity that inheres in the rules we follow cannot be understood on the basis of natural science.