When you find every Lost Word in a given book, you earn a reward:

The reward is granted automatically, as soon as you inscribe the final word. This reward varies from book to book, generally based on how hard it is to complete that book, and is displayed at the bottom of the page below the list of Lost Words.

Also, once you complete a book, it will henceforth appear in gold on your bookshelf.