The visitors equalised on 15 minutes as the ball came to David Norris midway in the Hull half and the Plymouth man ran 15 yards and fired in a shot from the edge of the box which flew into the corner of the net past Boaz Myhill.
Hull should have retaken the lead in the closing stages of the half as Sam Ricketts made a space for himself inside the box and struck a right-footed shot which keeper Larrieu parried right in front of Dean Marney.
He fired the ball right-footed from close range, but Larrieu managed to stretch and palm the ball away allowing his defence to clear the danger.
Plymouth were the team to take the lead however as on the stroke of half-time Gary Sawyer floated a ball in from the left and Rory Fallon rose to flick the ball past Myhill with a neat header from six yards.
Three minutes into the second half Hull were awarded a penalty as Peter Halmosi was adjudged to have brought down Bryan Hughes inside the box.
Windass took the penalty but keeper Larrieu was equal to it as he managed to parry the ball away, the rebound fell to Marney, however, and the Hull man put it in the back of the net, left-footed from close range.
Plymouth came close to restoring their lead soon after as Fallon found himself in space inside the box and curled in a right-footed effort from ten yards, but Myhlll did well to leap to his left and turn the ball round the post.
Plymouth took all three points on 81 minutes as substitute Sylvan Ebanks-Blake raced in on goal and struck a blistering right-footed effort past Myhill from just inside the box.