Xbox One, Xbox One S, and Xbox One X appear to cache textures and meshes more often during modded play than at other times. Sometimes, the PS4 will do this with vanilla textures as well. The build-up of these files can corrupt over time and cause gameplay issues and lag.
Clearing the cache or rebuilding the console storage database is part of good console maintenance.
So when should you do it?
When preparing your console for a new game.
Uninstalling a Game
After uninstalling a game you no longer wish to play.
If the console seems laggy or there are connectivity issues.
After installing mods with large texture files.
While playing with mods with large texture files.
As part of normal maintenance of your PS4 and PS5.
As part of normal maintenance of your Xbox.