(No, I didn’t take the time to cover the head-keeper-inner part. They only use it for such a very short time. Plus I’m lazy.)
If you are thinking of redoing your carseat cover, I say go for it! It took some time, but was generally much easier than I expected. Just take lots of pictures before you tear apart the old cover, and I would strongly recommend store-bought bias tape. The only imperfections on mine are because of my wonky hand-made tape. It just wouldn’t wrap all the way around the bulk of some of those 4 layer seam allowances.