Abstract: Galaxy formation is a complex, hierarchical, highly non-linear process, which involves gravitational collapse of dark matter and baryons, supersonic, highly compressible and turbulent flows of gas, star formation, stellar feedback, as well as heating, cooling, and chemical processes that affect the gas and, indirectly, the stellar and dark matter distributions. Nevertheless, despite the apparent complexity of processes accompanying galaxy formation, galaxies exhibit a number of striking regularities, such as tight correlations between galaxy sizes, masses, luminosities, and internal velocities and surprisingly tight correlations between properties of stars and gas in galaxies and the mass and extent of their parent halos dominated by dark matter. Existence of such correlations indicates that powerful processes operate to bring order out of chaos. Understanding what these processes are and how they operate is not only fascinating scientifically, but is critical for interpreting the avalanche of current and future observations of galaxies across cosmic time. I will describe recent progress in our understanding of how such regularities can arise in a seemingly chaotic and nonlinear process of galaxy formation.