Wittgenstein's Private Language Argument was received within the Pittsburgh School of Philosophy as raising new and interesting difficulties for the traditional empiricist model of knowledge justification. While McDowell's attempt to solve the difficulties in question is still debated, I claim that a different solution to the same difficulties might have been suggested by Wittgenstein himself in On Certainty.