Infants understand that people act in order to achieve their goals, but how can they tell what goals people find worthwhile? Here, we explore the thesis that human infants solve this problem by building a mental model of action planning, taking into account the costs of acting and the rewards actions bring. Consistent with this thesis, we found that 10-month-old infants, after viewing an agent approach two objects equally often, inferred that the agent preferred the object whose attainment required a costlier action. Infants’ responses generalized across changes in perceptual variables that distinguished one action from another (e.g. path length, angle of incline), suggesting that an abstract cost metric based on force or effort supported their judgments. These findings suggest that infants’ knowledge about agents may be expressed as a generative model for action planning, which can then be inverted to identify the probable hidden causes for observed actions.