I found when my imaginary world grew beyond one book that I might need some help remembering my own details.
This doesn't mean your story didn't make a lasting impression, even on yourself - this means your brain is working the way that it usually does and shedding minutae it is no longer immediately using.
My solution to this dilema is to use a small notebook. At each session I have it open next to me and I devote at least a page to every chapter. I write each page number down that I complete and keep a count for that section. If there are small details I think I might need I jot them down for reference.
I also have been known to draw crude maps of locations so that when I write about that same place on down the road I don't totally change where the front door was by accident...oops!
None of these things might work for another writer but they work for me and I developed them as I came along. If I help whoever reads this in any way then I consider that a bonus.