Not sure how much help this is...but here's what I do:
I use Calibre strictly to help me keep a running total of what books/authors I have/don't have and organize by my own set of keywords (I don't import the ones that come in the regular metadata but add my own). Once the books have been added, I delete the actual file formats from Calibre. I usually only use Calibre for the keywords when trying to find a specific book that I know I have but can't recall the title/author of.
In terms of how I organize the actual books, my top-level folders are Classics, Mystery, Romance, Comics/Graphic Novels, Gaming (RPG-related), Sci-fi/Fantasy, Non-fiction, and Hobby/Other. Under those top-level folders I have a mixture of category folders combined with specific author folders, depending on the number of books I have in that genre. I've always preferred to list authors by first name rather than last name as I don't think "Doyle, Arthur Conan" when I think of the author of Sherlock Holmes so I list my folder and file names the same way.
For example, under Classics my next level down contains folders for the B&N Classics, Delphi Complete Works, Fairy Tales, Legends, Myths, and Poetry as well for specific authors that I've collected most of the books by such as AA Milne, Agatha Christie, Alexandre Dumas, Arthur Conan Doyle, H G Wells, L Frank Baum, Lewis Carroll, Mark Twain, etc.
Authors that I only have a few books by are found within the Classics folder itself listed by author, such as Leo Tolstoy, George Orwell, Oscar Wilde and Mary Wollstonecraft.
So my folder structure would look something like (folders in bold):
Classics
- L Frank Baum
- L Frank Baum - Ozma of Oz (EMP).rar (contains all formats I have for that particular book, in this case epub, mobi, pdf)
- Mark Twain
- Mark Twain - A Connecticut Yankee in King Arthur's Court.epub
- Mark Twain - The Adventures of Tom Sawyer.epub
- George Orwell - 1984.mobi
- Jonathon Swift - Gulliver's Travels (EPLPr).rar (contains epub, pdf, lit, and prc formats)
Since it's rare for me not to remember the name of an author I collect (unless they are part of a multi-author series), I usually know immediately where I can find a book as soon as I know the author's name. If an author could fall under multiple categories - such as Agatha Christie who would usually logically be found under Mystery, I always put all books by an author in the primary category that
I think of them as. In this case, I tend to associate Agatha with the Classics rather than Mystery so I always know immediately where to look for her books if I need them.
It's not a system that would work for everyone but it works for me with a library of about 50k in titles.