Marcel is gradually forgetting Albertine. Time is the agent. He has a finite set of memories and as they fade they are not replaced, while he himself changes into another person.
It is not because other people are dead that our affection of them fades; it is because we ourselves are dying. Albertine had no cause to reproach her friend. The man who was usurping his name was merely his heir. We can only be faithful to what we remember, and we remember only what we have known. My new self, while it grew up in the shadow of the old, had often heard the other speak of Albertine; through that other self, through the stories it gathered from it, it thought that it knew her, it found her lovable, it loved her; but it was only a love at second hand. (V,805)
Swann is also being forgotten, but by a quite different process. Gilberte has been adopted by her step-father, de Forcheville, and wishes him to be known as her true father. She has inherited Swann’s tact and intelligence and Odette’s morality.
But when, to this daughter of his, he used from time to time to say, taking her in his arms and kissing her: “How comforting it is, my darling, to have a daughter like you; one day when I’m no longer here, if people still mention your poor papa, it will be only to you and because of you,” Swann, in thus pinning a timorous and anxious hope of survival on his daughter after his death, was as mistaken as an old banker who, having made a will in favour of a little dancer whom he is keeping and who has very nice manners, tells himself that though her he is not more than a great friend, she will remain faithful to his memory. She had very nice manners while her feet under the table sought the feet of those of the old banker’s friends who attracted her, but all this very discreetly, behind an altogether respectable exterior. (V,800)
Gilberte’s presence in a drawing-room, instead of being an occasion for people to speak of her father from time to time, was an obstacle in the way of their seizing the opportunities that might still have remained for them to do so, and that were becoming more and more rare. Even in connexion with the things he had said, the presents he had given, people acquired the habit of not mentioning him, and she who ought to have kept his memory young, if not perpetuated it, found herself hastening and completing the work of death and oblivion. (V,800)