We will discuss the application of multi-scale graph theory based methods to the detection of general structures in networks, lattices, and amorphous physical systems. These methods enable the detection of the "natural" system structures on all scales. We specifically analyze lattices and spin systems with defects and various glass formers (including an analysis based on experimental data) to ascertain dominant structures at different temperatures. We will discuss general features of the phase diagram related to this analysis