In his book Making the Social World, John Searle gives voice to an aspiration that is very popular in social ontology. This aspiration consists in the notion that social objects can be reduced to physical objects. From this perspective, social objects are nothing more than physical objects on which we impose functions that are merely subjective and therefore external to them. Now, my paper has two objectives. In the first part, I want to show that Searle’s account entails some sort of interpretationism that gets into difficulties once we reflect on his essential notion of imposition. In the second part of my paper, I want to sketch a Wittgensteinian point of view that allows us to describe our relation to social objects without being committed to the flawed model of interpretationism and to its metaphysical assumptions.