Three experiments investigated the spatial frames of reference used in memory to represent the spatial structure of the surrounding environment. Participants learned the locations of objects in a room. They were instructed to learn the layout along an intrinsic axis, which was different from or the same as their viewing perspective. Objects were placed on a rectangular mat oriented with the walls of a room or on the bare floor of a circular room. After learning, participants made judgments of relative direction using their memories (e.g., Imagine you are standing at the book facing the lamp; point ot the clock). Pointing judgments were faster and more accurate for imagined headings aligned with the learning axis, even when it differed from the viewing perspetive, and there was no cost to learning a layout along a nonegocentric axis. If the shape of the layout was "square" relative to the intrinsic axis of learning, novel headings orthogonal to the axis of learning were retrieved more efficiently than were other novel headings. These results suggest that spatial memories are defined with respect to intrinsic frames of reference, which are selected based on egocentric experience, sopatial and nonspatial properties of the objects, and cues in the surrounding environment.