I had the lazy pedal return after working on the front calipers. Didn't have the hydraulic side apart (other than bleeding when I'd done) and all I really did that the system could see was push the pistons in and pump them out again just to check they weren't seized. When I was finished (after bleeding at the calipers) lazy pedal return and long mushy pedal. (At that point I only bled the fronts and only because I was 'there'. Bleeding the rears recently has massively improved pedal feel). It came alright on its own but I did bleed the little fitting on the ABS unit. Mine has Hydraboost - not vacuum servo.
Is there any other work you have been doing on the brakes?
FWIW, a vacuum servo with the engine off should deplete its vacuum in around three (maybe more) pedal pushes and the pedal be solid from then on (and the vehicle damned near unstoppable if being driven) and when you then start the engine with the pressure on the pedal, you should feel the pedal move from under your foot.