Chug asks me:
Tyler, do you scrawl notes in the margins of the books you read?
No. The key constraint for me is finding the page, not remembering the associated idea, and I find it odd that other people are not also this way. (That said, I do know I am the outlier. But really: do you write the notes to actually remember something? Or do you do it to make the reading experience more real in the first place, much like taking notes in class?) So in some of my books I dog-ear the pages which are important for me. In other books I write those page numbers on the inside book jacket.
In the longer run I expect "annotated" books will be available for full public review, though Kindle-like technologies. You'll be reading Rousseau's Social Contract and be able to call up the five most popular sets of annotations, the three most popular condensations, J.K. Rowling's nomination for "favorite page," a YouTube of Harold Bloom gushing about it, and so on.