“In Revelation 4:8, it says the four living creatures do not rest day or night worshiping God. But in Revelation 22:5, it says that there will be no night there. Will there be ‘night’ in heaven, or not? Is this a Bible contradiction?”
There is no contradiction. The passages are using descriptive terms that communicate different points. Revelation 4:8 describes the ongoing, continuous worship of God. To make that point, the apostle John describes it as going on “day and night” (New American Standard Bible), that is, it is occurring constantly. If John was writing today, he might say they were praising God “around the clock”. Although in heaven there will be no clock, it still describes the constant worship that heaven offers to God.
The second passage is talking about a different aspect of heaven. It is a place filled with God’s glory. . .
“And there will no longer be any night; and they will not have need of the light of a lamp nor the light of the sun, because the Lord God will illumine them; and they will reign forever and ever.” (Revelation 22:5)
This verse is not speaking of the unceasing nature of heaven, but of the nature of God in heaven. There will be no night because God is there. Where God is, there is light (which drives out all darkness).
John is using a metaphorical phrase to describe time – “day and night”. He is not being literal in your first reference. He is merely conveying the non-stop worship of God. But heaven will be full of light constantly, because God will be there.