We present a theoretical model that formalises the notion that an individual's preference for a particular behaviour (e.g., cooperation) can evolve as an adaptation to her social environment. At each point in time, players from a population meet to play a game. The game is drawn from a family of 2by2 symmetric supermodular games according to a distribution, termed an environment. Players' ability to evaluate which action is optimal is limited, but they make fewer mistakes when the payoff differences from the two actions are larger. We find that preference biases towards one of the actions are fitness-maximising given the existence of mistakes. In environments in which payoff differences between actions 1 and 0 are higher (in a first-order stochastic dominance sense),stronger preferences towards action 1 evolve. Such biases do not evolve if players evaluate actions perfectly. When applied to a two-strategy reduction of the indefinitely repeated prisoners' dilemma, our model predicts that environments with higher distributions of the continuation probability lead to stronger preferences for cooperation. The model sheds light on how cross-culture variation in experimental cooperation rates relates to differences in common practices of these cultures.