Strangely enough, we think at least some of those days, he was in Toledo ... who knew? Seriously, though, the Catechism of the Catholic Church directly addresses this in sections 631-637. I will summarize as best I can what is found there.
First, we need to understand that Jesus did, in fact, die. “... Jesus, like all men, experienced death, and in his soul joined the others in the realm of the dead.” What happened to Jesus on Good Friday wasn’t something akin to death, but actual death. That is important because of what that death allowed Jesus to do. As the Son of God, he descended into death as the Savior, not just another person who died. The idea is this: that all death that occurred before the “Jesus event” led people to an afterlife called Sheol, where they were “deprived of the vision of God.” This was the case for both the good and the bad who had died. However, we do know that the afterlife was different for the good and the bad, as is shown in the story of Lazarus. Jesus said Lazarus was at “Abraham’s bosom,” and the rich man was “in torment.” (Lk 16, 22-26)
So, before Jesus, everyone who died descended into Sheol; a place where they could not see God. The people who rejected God had a different experience there than those who followed and loved God. When Jesus died, he descended into that place and delivered those who knew, loved and served God from Sheol into heaven. Remember, Jesus “did not descend into hell to deliver the damned, nor to destroy the hell of damnation, but to free the just who had gone before him.” (CCC 633)