In the name of psychoanalysis, for more than one century, various theories have been produced about non-binary gender and sexualities, evaluated with respect to their conformity to the Oedipus complex, the castration complex, or the formulas of sexuation and the phallus. Most of these analyses refer to the Symbolic, a Lacanian or Lacan-inspired register of transcendental, universal rules that determine subjectivity and the "normal" processes of subjectivation. This raises the question of the effect of a structuralist conception of the Unconscious on human lives, and on the constitution of "lives worth living". For many psychoanalysts, Merleau-Ponty's conception of the Unconscious proves misconstrued insofar as he leaves aside its linguistic dimension. Hence, to which extent may a non-language-based conception of the Unconscious avoid imposing universal subjectivation categories that stretch beyond history and culture? The aim of this article is to show how Merleau-Ponty's non-exclusively-verbal Unconscious is concerned with affect, a perspective that entails a different, not exclusively linguistic conception of the Symbolic. For that purpose, I argue that Merleau-Ponty's phenomenology, differently from Husserl's, is a phenomenology of affectivity. Expatiating on the centrality of the lived body, it moves away from constitution to institution, in a way that places affectivity, and not language at the core of the Unconscious. This conception results in two consequences: the Unconscious is not, or not entirely verbal, which implies the possibility of a non-binary reading of gender and sexuality. This non-exclusively-verbal Unconscious may lay the ground for a feminist analysis of gender.