Javert knows about sin. Raised in the orphanage, he could not escape it. Everywhere he looked, he saw the whores and their bastard children. He became a gendarme in the hope of turning at least one life from its wicked path.
He knows that Valjean is a sinner. The law of God dictates that Christians should follow the laws of man. And that means taking the punishment that you are given. Not running off to take a different name and becoming Mayor!
But he knows that the greatest sin of all is his own feelings for the man he pursues.