Noble metal nanoparticles that support localized surface plasmon resonances (LSPRs) have the unique ability to manipulate and confine light at subwavelength dimensions. Utilizing these capabilities in devices and coatings requires the controlled organization of metal nanoparticles into ordered or hierarchical structures. Polymer grafts can be used as assembly-regulating molecules that bind to the nanoparticle surface and guide nanoparticle organization in solution, at interfaces, and within condensed phases. Here, we present an overview of polymer-directed assembly of plasmonic nanoparticles. We discuss how polymer grafts can be used to control short-range nanoparticle interactions that dictate interparticle gap distance and orientation. We also discuss how condensed polymer grafts can be used to control long-range order within condensed nanoparticle–polymer blends. The assembly of shaped plasmonic nanoparticles that have potential applications in enhanced spectroscopy and optical metamaterials is highlighted. We end with a summary of promising new directions toward the fabrication of plasmonic nanocomposites that are responsive and possess three-dimensional order.