Consistency is an important UX heuristic, and it helps lend visual harmony to a set of similar objects.
It's helpful to ask: What's the job each of these cards? It seems like you want to be able to quickly browse what you've read, find a particular book, and remember what you thought about it.
As you're finding out - trying to put the entire review on a card is leading to wildly inconsistent visuals.
But without content, some of your cards are looking too blank. How do you solve giant discrepancies in content amount?
Well, I think showing a picture of each book's cover will speed up the task of finding a book, because we don't always remember exactly what a book was called, but we usually remember the book as a visual product. I know you're aiming for minimalism, but you can still achieve clean layouts with consistent book cover visuals and lightly-designed cards.
Under the picture of the book, you can add the title, star rating, and a brief, truncated intro of the review. To read the whole review, simply tap the book, and open the side panel you're working on.
Here's a nice example by designer Davide Galbiati:
