A fundamental unsolved problem is to understand the differences between inanimate matter and living matter. Although this question might be framed as philosophical, there are many fundamental and practical reasons to pursue the development of synthetic materials with the properties of living ones. There are three fundamental properties of living materials that we seek to reproduce: The ability to spontaneously assemble complex structures, the ability to self-replicate, and the ability to perform complex and coordinated reactions that enable transformations impossible to realize if a single structure acted alone. The conditions that are required for a synthetic material to have these properties are currently unknown. This Colloquium examines whether these phenomena could emerge by programming interactions between colloidal particles, an approach that bootstraps off of recent advances in DNA nanotechnology and in the mathematics of sphere packings. The argument is made that the essential properties of living matter could emerge from colloidal interactions that are specific—so that each particle can be programmed to bind or not bind to any other particle—and also time dependent—so that the binding strength between two particles could increase or decrease in time at a controlled rate. There is a small regime of interaction parameters that gives rise to colloidal particles with lifelike properties, including self-assembly, self-replication, and metabolism. The parameter range for these phenomena can be identified using a combinatorial search over the set of known sphere packings.