The influences of microstructures on the yielding and strain hardening characteristics of carbon steel are examined in the framework of computational homogenization. We first conduct the tensile tests on two kinds of specimens; steel with pearlite and spheroidized cementite structures. Then the numerical two-scale analysis is performed to simulate the tensile tests by the global-local analysis method. Using the numerical results, we clarify the micro-scale mechanisms, which characterize the macroscopic plastic behavior. Finally, the multi-scale inverse analysis method is proposed to show that the experimental facts entail the higher yield stress of ferrite (Fe) for pearlite structures.