“It’s turtles all the way down!”, according to legend, is what an old lady said to a physicist after hearing him lecture on cosmology, refusing to give up her traditional belief that the world rested on a turtle, which rested on another turtle, which rested on another. The expression has come to stand for an infinite regress. If something relies on another thing, which relies on another thing, at what point is there a final turtle that relies on nothing? Here we have the opposite problem: In order to halt an infinite regress, we must engage in circular reasoning and declare a final thing to be self-supported.
In his Philosophical Investigations, Wittgenstein poses the problem of the child at the blackboard. If a child does not understand how to do arithmetic, we can teach the child rules to show them how arithmetic is done. However, what if the child does not understand the rules? We could teach the child rules for understanding the rules, but this leads to an infinite regress, to a series of turtles that seems to recede from sight. At what point does the child understand rules of the rules? What rules require no rules to teach the child our interpretation of arithmetic, such that the child can practice it the way that we do? The mysterious answer, Wittgenstein suggests, is that our practices do not rest on rules. It is not language that is at the bottom of our behavior, such that we learn rules to engage in practices. Rather, we learn practices by imitation, largely without need for words or explicit rules. We only need rules to guide the child, who is already engaged in practice, as signposts to guide the child into correct practices rather than incorrect ones. Language, like the turtle, is only one element in the situation, as are human judgements. The world in which we live goes beyond the limits of language and rules. Experience cannot be fully fleshed out in language, nor can practices be fully articulated by rules.