Then YHWH said to Abram, "Know for certain that your offspring will be sojourners in a land that is not theirs and will be servants there, and they will be afflicted for four hundred years."
Genesis 15:13

God said that the Hebrews would be sojourners for 400 years. That entire time wouldn't be spent in slavery in Egypt. The 400 years starts at the birth of Isaac and continues until the Exodus. They were slaves in Egypt for much of that time, but also sojourners in Canaan for many years.