A rational agent is someone who knows what she is doing and why; and we hold her responsible for her actions. However, in our everyday lives we often act automatically and even involuntarily. The aim of this paper is to motivate a reconsideration of agency such that we can conceive of basic forms of the unconscious, the involuntary and the unreflective as being part of human responsive agency. The paper dwells on the structure of self-experience in habits to reveal how temporal displacements, disintegration and self-alienation are part of human self-experience. The central claim is that the responsive structure of habits form a possibility for re-appropriation of our own actions. This will allow us to acknowledge that a large part of our actions and decisions are less rational and deliberate than we might have hoped, while avoiding the unfortunate conclusion that such behaviour is impersonal and simply a complex form of reflex.