Fillet weld joint is widely used in engineering structures, but a lot of failures have been generated in the fillet joint affected greatly by weld residual stress, and it is very important to decrease the residual stress. Therefore, this paper proposes a new method using overlay welding and cutting (OWC) to reduce the residual stress in the fillet weld. First, the overlay welding is applied on the root surface of fillet weld, and then the overlaid metal is removed again by cutting. In order to verify this method, a thermal-elasto-plastic analysis method, using finite-element analysis (FEA) techniques, is developed to evaluate the residual stress change during the process of OWC. The impact indention measurement is also used to measure the surface residual stress. The results of FEA were compared with experimental data to confirm the accuracy of the developed finite-element method (FEM). In order to provide a guideline for design, the dimension effects including overlay weld width and height on residual stress have been investigated. It finds that OWC can decrease 25–40% of the as-weld residual stress, and increasing the overlay width and height is helpful to decrease the residual stress, which provides a reference for the reduction of residual stress in the fillet weld.